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ABSTRACT , 

This "study Considers whether or not the various 
sentence structures in standardized reading tests cohere to 
facilitate the extraction of Information. In the context of the 
theoretical framework of the study, a schema theory of reading, a 
dif f us^discburse structure may legitimately activate more than one 
way of cfcrreeptualizing an answer to a test question. The methods used 
to explore this issue were the collection of a set of clinical 
interviews with children: in grades 3, 5, and 8 in the Mohtgomery 
County (Maryland) Public Schools who h?d taken a standardized 
achievement test; and the administration pf an experimental test. The 
interview data indicated! a strong probability that test performance 
could be influenced by thfe structural organization of passage, 
information. Analysis indicated that low-achieving students lacked 
metacomprehension and test-taking skills, not comprehension ability. 
An experimental test was designed to reduoe ambiguities, 
inconsistencies, and inaccuracies in the passages -and questions in 
order to assess more accurately students' comprehension ability. An 
important result was that the main difference between good and poor 
readers involves a reluctance o*r inability on the part o^ poor 
readers to process information hierarchically. (AMH) 
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Preface ^ 

Notwithstanding the title of this report and its failure to mention 
reading instruction,, the conception and design of this- study originated 
in a practical educational problem. How closely did the demands of a 
particular reading comprehension test match up with the 4osands and 
objectives of a particular reading curriculum? The curriculum in question 
was the Instructional System in Reading/Language Urts of the Montgomery 
County (Maryland) Public Schools which over the years of its development ■ 
has stressed the importance of detailing objectives based on trie discourse 
properties of* reading passages. This is an innovatiye approach to 
reading curriculum development and one which it was not certain vduld 
'align with the objectives, of standardized reading comprehension tests; 
The test objectives, while admittedly not ignoring discourse concerns, 
are broadly stated in terms of modes of processing* which presumably- 
crosscut the -various types of discourse included on a reading test. 

sit became clear quite early into the investigation that the MCPS 
objectives were not being measured on the standardized test— a not 
surprising conclusion in itself, since the intent of 'the tests was at 
variance with the intent of the curriculum. There is, of course,, no 
value judgment that can be made on the simple evidence of a mismatch. 
However, in honing the analysis, there was reason to suspect that' the 
tests had invalidated certain questions by not attending to matters of 
discourse organization. 'In other words, there were indications that 
a child taking the test might Legitimately see more than one right answer 
or might 1 legitimately eliminate the correct choice. 

What constituted a 'legitimate 1 problem, however, was at % that stage ' 
of the inquiry a matter of some uncertainty because the theory capable 
of informing a sur*e jifdgment was itself imprecise. I* was in the' interests 
of contritjuting to the clarification of this theory that the present * 
study was proposed, to the National. Institute of Education. The intention 
was to expand the scope and purpose of the MCPS investigation, first to * 
subject several reading tests to a fine grained linguistic analysis, 3 
^focussing especially on discourse propertied, to question test takers 
about their behavior and impressions on a test they were required" to take, 

0 

and-, 'last, to formulate arjd test a set of hypotheses ..about what con- 
stitu^d imprecision and difficulty in the tests. The testa^ selected 



were the California Achievement Test (CAT) '(1978 edition,- Forms C and D, 

Montertm^CA:* CTB: McGraw-Hill j , the Metropolitan Achievement* Test (MAT) 

(1978 edition, Form JS, New York: .The Psychological jCor proration) , and 

the Iowa Test of* Basic Skills (ITBS) (1979 edition, Forms 7 and 8, Iowa 

City: Houghton Mifflin), ' " * ' 

Th6 study was' carried out, where it had been initiated, in the , ^ 

Montgomery County Public Schools, and I would lake to express my gratitude 

to the district; personnel arid the principals of the schools cooperating 

in the study for* their interest and hfelp. I am, expecfially thankful to 

Ted Schuder and his- staff* in Instructional Development for paving the 

v - • 

way, landing an 'ear, and benfdtting me with their breadth of experience 

in reacfing and testing and v to Susan Gross in Accountability for her ^ 
help with the data analysis. * 

• The project was implemented with the invaluable assistance of Joan 
Koppelman, who Consistently arid, with dedication pe^fbnnecl beyond my - 
expectations. Bfepides' skillfully managing the considerable number of 
clerical chores of the project, she acted, out of her background as 
a teacher, as a valuable brake on my .speculation. -She came dltimately 
*to have an important influence on many of the "ideas- which bec'aroe 
embodied in this report, in Joan's absence during the 'preparation of 
this report, ♦Donna Cjiris£ian_jua£l ifyby Berkmeier ably ancf w±^h con- 
siderable patience tooki'on the burden of h formatting, proof irtg, find 
typing, for jtfiich I* am deeply appreciative and in their debt. * 

I owe, of course other debta of a less tangible nature to other 
investigators in "reading and testing. I v have .acknowledged some,, of these 
In the context of the background discussion, but many others should have 
been included. It Was in the interests of keeping the discussion as 
focussed as possible on the -documentation of the study that gheater* • t 
consideration was not givgn to more of the relevant literature. Beyond 
„ this consideration, Jsoyever, a good deal of pertinent research — * 
in particular, a study be£ng conducted* by 'Paul Kay%and Charles Fillmore 
(see Fillmore I982)-*is sfeilL in progress, a*id .a comparison of findings , 
would have been premature. In later' publicaitons and as appropriate 
this vJo'rk wil^. be properly referenced and discussed. Hopefully, by that 
time, this report, will have accomplished its interim* task of 'encouraging - 
the kind^ of critical response jiefed^d 'to sharpen its presentation Und 
expand the scope of it$ educational implications^.* 
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INTRODUCTION 

The icieal test knows in detail, what it is doing. It knows that 
its questions fall withito the range of experience of the people who 
will take it, it understands the perceptual and cognitive demands of 
its questions, it encourages only legitimate applications of its 
results, and it , looks like wtj^t it claims to ber^Collectively ;the6e 
considerations constitute matters of test validity, reassuTlhces that 
the test does, Jin fdct do what it claims to do. The f^rst consideration, 
fqr example, concerns the matter of content validity . The test questions 
requir^ 'answers which the test takers can with some probability be 
expected to know. ""The setond consideration is mor^pbstiract and harder 
to discern. It assumes that the Cest has been constructed in conformity 
"with genbrall^, held beliefs about how similar information would be 
processes \n a non-test situation. Furthermore, it assumes that the 
test has-been fairly constructed, at a minimum presenting no' ambiguous 
qv misleading alternatives. Considerations of this kind are encompassed 
in the te^m construct validity . The third consideration requires that 
the test ijoake claims about its 'usefulness that are consorfant with its 
objectively determined value. This requirement in effect restricts 
tfle scope jof predictidns that can be made'about the test taker f s behavior 
on the basis of test results. This consideration is generally referred 
to" as predictive validity . The last consideration constitutes the 



weakest demdtistration of Validity. It requires only that the test be 
formatted so that test* takers recognize it as a test of what it claims 
tO;be, a natter of what is generally called face validity . 

"A "carefully constructed test instrument will tal^e pains to 
demonstrate to users* that it has satisfactorily- addressed each of the 
validity conisidejrationfe. Part of this demonstration typically entails 
the presentation of evidence that the test on repeated applications 
achieves* statistically comparable results, an indication*of its 
reliability £s a test instrument. A reliability check, howevfer, does 
not of itself fully answer questions of validity. Most importantly, 
we would ^till not know tfiat the test actually measures what it claims 
to. St migh>t be . reliably me^ptiring. some aptitude or behavior which the 
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test makers had not anticipated., In other words, although a reliable 
t|st has validity for some purpose!/ it ma^ not be the purpose that the 

test makers have in mind. 

' » * * 

Checks on test validity typically involve a correlational analysis/ 

For instance, if the concern^s the content validity of a 'reading tefet, 

the test developer might want to show that the instrument resembles in 

important respects Jthe curriculum .used to teach children to read. If 

the concern is* the test's predictive validity, the ^est makers would 

look for strong correlations between test performance and future academic 

success such as grade advancement or school leaving. If the concern is 

face validity, the test maker might! want to show that test results" 

correlate highly with results on otjier tests intended for the same 

purpose. 

Checks for construct validity are probably more difficult to make , * 

\* • ■ 

than other validity checks, although in principle the methods are 

similar. TTle test maker still wants to find a correlation, bijjt searches 

/or one between the skills used in salving reading test problems and 

" the skills "used ir> other reading situations. What test makers usually 

do is engineer the test in such a way "that individual questions address > 

particular test objectives, these being a set of comprehension or , 

prob lei/^olving skills. The Iowa Test pf Basic Skills (ITBS), for - 

instance, organizes its reading comprehension subtest around sixteen 1 • » 

skills, - each falling within on$ of three more inclusive catdgorie's. 

For example*, individual questions are determined to reveal information • v 1 

on the test taker's ability to understand factual details xelating to v 

classification, a literal meaning "ski^l; to draw 'concisions from in- - m : 

formation and relationships t an interpretive meaning skill; or to \ 4 

recognize the main idea or topic of a [paragraph or' selection an, 

■ i , 

evaluative meaning skill. Thirteen other Skills are tested in addition.. 

/ v I / ' * « " *' 

fcven from this sketchy description it is apparent that the 'delirheatiQn 

of skills objectives and the demonstration of construct validity are 

intended to be useful for classroom instructors and' curriculum developers. / 

If a test lays claims to being a instrument able to huelp. eciuc*ators' in^ 

making instructional decisions, the worth of the claim lies sub- 

' stantially i n the 'demonstration of *the test's construct Validity. 
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The other demonstrations of validity are perhaps ^>f -more 
importance to administrators. Issues revolving around content validity, 
for instance, ultimately affect decisions ^bout whether to favor one 
test over another. Obviously if a test is constructed along the same 
lines as the curriculum,- it will be to the school* s advantage to select 
it.' over another test constructed along Afferent guidelines. Then too, 
if the^ school^ is primarily interested in ilsing the test results to 
aid in student placement decisions or to compare student performance, 
more decision Weight might be given to the demonstration of predictive 
validity. 

The complexity of validity considerations leads us naturally 
enough to question how much any one test can hope to accomplish at\d 
ho^ many.' interests it can hope to serve. Many researchers in the* * 
testing field have pointed out the trade-off which must be made between 
a test capable of discriminating individuals and a test capable of 
diagnosing individual strengths arid weaknesses. The problem centers 
around the point at which a particular test question becomes difficult 
to a majority of test takers. For purposes of comparing and ranking 
a set of test takers, it is best to have the average difficulty of test 
items' center around 50%. At this rate half the, test takers would 
answer the item correctly and half incorrectly, and the performance of 
all test takers would approximate a normal distribution. In this way, 
the test can maximally discriminate among all test takers J At a 
higher difficulty level the te6t has lesser power to discriminate among 
poor achievers, while at a lower level it loses its ability to dis- 
criminate among, higtt achievers. On the other hand if ^tbe purpose of 
the tefet is. not to compare test, takers but to evaluate the standing * 
of each test taker against some desired standard, say satisfactory 
completion of 80%^t the test items, then difficulty levels of individual 



_ • \ 

items must be J lo^^M,so that the test as a whole approximates tHe 
standard in difficulty. The logics dictates that a test designed in such 
a way that half of the students correctly answer less* than half the 
items has little power to inform evaluators about the true or absolute 
achievement of individuals. The logic is predicated on .a culturally 
based assumption that a group level of achievement no higher than 50% ' 
borders on being deplorable and indicates an unacceptable level of 



* effort. If the test has an educational intent and y^roup performance 
is as low 50%, someone invariably has' to provfSe an explanation of 
why performance is not higher.' #w * 

Up until 20 years a£o educational testing in this country con- 
sisted primarily of teacher-made tests, seeking to revjeal children* s % 
understanding of ^heir instruction, and standardized noifra-ref erenced 
\ te.sts^ seeking to inform teachers and parents about how well children 
were doing. in comparison to one adother. The N teacher-made tests,, 
being specific to fcjie instructional goals of' a single teacher, were 
not subjected to validity or reliability checks. It was rtecessary to 
accent their validity on .face value, trusting to the integrity and * 

* credibility of the teacher. The standardized tests, with their aim 

t set at the performance of students across the nation, obviously could 
not be tied closely to the instructional objectives of any one - * 

curriculum. To demonstrate their applicability to a wide ranging 
set of users, careful checks on reliability and validity had £o be con- 
ducted. TJie effect of these checks was to confer on the tests the , 
impression of objectivity, a distinction which the teacher-made tests 
might claim but could' not prove. Over time this led educators concerned 
about the accountability -of their actions to put greater reliance on 
the standardized tests and make them serve double duty as diagnostic J 
ancPplacement tools. From the standpoint of test theory, however, 
this extension of scope was questionable since by design h^lf of 
students taking the test would score below average^and would look like 
they needed remediatidfc. But do they in fact need remediation or is 
Tt~"just that the score ofc the average achiever is set unrealistically 
low? • 

t 

Possibly on theoretical grounds alone, but more likely because of 
the potential for abuse which arises when an instrument ds used for 
an inappropriate purpose, a reaction set in against the use of^ 
standardized norm-referenced tests for making decisions about individual 
students. Charges ^*ere brought that the t^rts were culturally biased, 
that they were tailored for middle cla§s white American values and 
experiences and that they discriminated against members of other groups. 
Other charges were made that the^*«^ts were being administered to 
groups who were excluded from th^norming sample — in a documented case 
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Spanish speaking students with limited English ability— and that the 
, test results were used as justification for placing, these students in 
classes for the mentallyVetarded. Overall hcfwever, the"^ajor 
criticism came to be that the label a student acquired because of * 
performance on a standardized test endured, not so much because fchere 
was truth in it but .because the placement decisions made for the 
student perpetuated it. Students said to be below average are taught 
as if tfrey were below average; Over time they come to believe the 
label and set their aspirations in accordance with it. The feeling is 
growing that too much import hangs on the student f s performance on a 
single achievement test, especially given that factors such as test 
anxiety, motivation, interest, and expectation also enter into the 
equation which produces the final score. 

With standardized tests as influential as *they are in informing 
educators, it is critically important to view them as objectively as 
possible and to evaluate the strength of their claims against a reasoned 
T and balanced assessment of their design and implementation. In the 
study reported here we will look at a fundamental issue on which will 
turn a judgment of the construct validity of standardised reading 
comprehension tests. Does -theJj-nguistic structure of reading tests 
influence the comprehensibility of test items? This question has a 
trivially affirmative answer under the meaning that any test passage 
or question must conform to an acceptable standard of grammaticality . 
It is not, hoyever, this level of analysis which primarily c<mc£rns us^ 
On the surface at least, most sentences in reading tests are well- 
formed and structurally, intact. Less concedable is whether the various 
sentence structures cohere to facilitate the extraction of information. 
It is possible that they may be so disjointed that information integration 
is impeded'. In the #ogic of the schema theory of reading adopted 
here (Chapter 1), a diffuse discourse "structure may legitimately activate 
more than one way of conceptualizing an answer to a test question." If 
this happens then skills other than those strictly involved in reading 
comprehension are involved in selecting the 'correct' or expected answer. 

In Chapter 2 a discussion of the methods used to explore this * 
issue are,, described. They include two major procedures, the first a* 
set of indepth., clinical interviews with children in the 3rd, 5th, and 
8th grades and at two proficiency levels. , The point of the interviews 



is to determine how and" why children taking standardized reading compre 
hens ion tests answer the way they^do. The results of the interviews 
are discussed in Chapter 3. 

The interview data, although it proved valuable -in its own right, 
was originally, intended to serve as a conduit to an experimental 
procedure by 'suggesting ways in which test passages could be rewritten 

4 I 

to enhance their compreljensibility and improve test performance for 
low achieving students* The interviews did in fact indicate a string 
probability that # test' performance could be influenced f>y* the structural 
origanization of passage information and the experiment was designed 
and executed as planned. The results of it are presented in Chapter 4. 

The sum of the information presented in this report strongly 
Suggests that greater attention be paid to the design, of reading . test . 
passages knd Questions. N Two points in particular ar£ argued: the 
tests in the form they now exist are inadequate to assess the true 
comprehension skills of the children who take them; moreover, they • 
fail to assess the actual proficiencies by whfch children achiever or 
fail on individual items* Each of these claims constitutes a serious 
doubt regarding the claimed coftstruct validity of the reading tests. 
A more detailed synthesis of the argument and its ramifications- is 
presented in Chapter 5* 



• ' CHAPTER 1 * ■ 

THEORETICAL FRAMEWORK 

The choice of an /appropriate level at which to conduct -a 
^linguistic analysis of reading tests has to be motivated by the full 
variety of objectives which,, the tests purport to treat. Since many 
of these dbjectives presuppose a data base of greater extent than a ' 

singie sentence, it would be advantageous to direct attention to the 

* r r f f 

linguistic structures which characterize the discourse or text as a 

whole. 'This point is h§rdly argued anymore. Most researchers in read- J 



ing and testing, in fact, <fc concern themselves with discourse process- 
iq^^ssues, although the?e is no" large scale agreement on vhat constitutes 
a viable approach ta discourse analysis or even on how to define 
discourse (Winograd; 1977) . The'dif f erences result from researchers 
beginning their analyses from different points, sota^ from* the level of 
the sentence working and others from, the level of discourse purpose 
working down. Part of the difficulty stems from the wide scope of 
discourse, which as'a field encompasses the full breadth of concern 
for meaning in language. Discourse analysis, consequently,', has been 
broached by rhetoricians, linguists, philosophers, folklorists, anthro- 
pologists, psychologists, sociologists, and computer scientists, each 
group addressing the concerns arising from its own discipline and 
applying methods of analysis which are favored or even idiosyncratic 
to its .own field. The full integration of these various approaches 
has not as yet been accomplished and as a tesult we are still operating 
without A comprehensive discourse theory. 

It was beyond our scope to engage the* full range of inter- _ 
disciplinary issues involved in discourse studies, although we did 
consider those issues which collectively would lead us to foqmulate 
a working definition of d^s'course.* The major desideratum was Co 
characterize discourse as a s6lf -contained ^structural entity not 
defined simply as an aggregate pi sentences or as ait^xemplar of some 
communicative intent. The proper defining characteristics were felt 
^to be those structure-to-function analogs from which writers selecr 
to give their accounts coherence and cohesion, to achieve their 



ptlrppses in writing, and to satisfy the needs of the reader. Implicit 
in this characterization are the existence o'f an interpersonal dnter- 
action between the reader and the writer" and, a constructive inter- 
action between the reader's background information and the new infor- 
mat ion in the passage. / 

Writer-reader interactign 

lb certain respects the writer-reader interaction has character- 
istics in common with conversational interactions, differing* mainly 
in that the writer and the reader are" not in temporal or physical / 
proximity at the time of the communication. ^JThe commonalty of the - 
tvo^vents hingfes on the structural implications which follow out of. 
a need for the communicants to be cooperative. Grice (1^^) first 
described these conversational implicatures, as he calls them, l as 
predicated on • Cooperative Principle , whereby each participant recognizes 
in a conversational situation "a common purpose. . .or .. .a mutually 
accepted direction 11 (45). The stated or implied agreement on the 
purpose of the communication makes judgments about the acceptability 
or appropriateness of individual remarks possible. If participants are 
cctonhunicat # ing in good faith , then both assume that any remarks made are ^ 
implicated in the purpose of the conversation. In other -words, every-^ * 
thing 'that is said ds expected to have some discerpible relationship , 
to the reason the principals are communicating. 

- \ We assume that the same postulates hold far a written communication, 
although the interaction of the writing situation is necessarily less 
dynamic than its conversational counterpart. In a-^ace to face con- 
versation > the .information exchange can^be individually tailored to the 
. expressed needs a^d informational backgrounds of both participants, 
while in a. writing situation' writers may not know with certainty who 
their readers will bie. „ |Fhey have to anticipate the information needs 
and backgrounds of possibly a wide variety of readers, who each may 
be reading for different purpqse^.^ Conversational encounters typically • 
^clude structural ^devices whose purpose is to assure that both 
participants are perceiving each other s infbrmation equivalently . 
*Ir v ope of the participants has difficulty identifying the function of * 
a particular utterance , the other participant can interjrupt^the flow 
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'of the 7 conversation to its detennined^fck>se long enough to show the 
relevance of the remark, Keenan and Sch^BEelin (1976) demonstrate how 
this procedure works even at fundamental leveis, for instance in 
situations where both participants must assure each other that they • 
have the same topic referent ix\ mind* # Writers are less able to. resort 
to such devices because they are providing a monologue account and 
because,' not being in face to face contact 'with their communicants, 
they lack immediate feedback data from non-verbal signs (Winograd 1977) . 
The puzzled looks, squirming, and signs of^inattention, which would 
otherwise inform them about hpw well their material was being compre- 
hended, are absent. The decisions that a writfcr makes must then be 
more considered or planned (Ochs 1979) if they are to be effective " 
in 6ommunicating the^intended message. 

But in what does the planning consist? What sorts of criteria • 
must the writer turn around mentally and transpose onto paper to make 
the message salient and clear to the reader? Intuitively, approaches p 
which take no ^recognition of the purposes f or ^ich people engage in 
literacy events or which take no recognition of the structure in 
which information is conveyed cannot provide the answers to these 
questions. The answers wj.ll be found at a level of k analysis where 
structure intersects function and where the continuity of the message 
can be detected. The appropriate starting poibt of the study was 
therefore determined to be neither at the level of the sentence nor at 
£he level of the discourse as a whole, but at the intermediate level 
of the paragraph. 

The paragraph as a unit of discourse 

* The status of the paragraph as an organizational entity has been 
argued primarily out of a rhetorical tradition. Rhetoricians, however, 
have almost exclusively approached the paragraph from the perspective 
of function and style. * / 

Their concerns have been with optimizing information flow; 
describing how ide^s can be presented to get a message across with no 
loss of accuracy. They have paid lesser attention to the linguistic 
mechanisms underlying the accomplishment of these goals. Consequently 
the terms which have been elaborated out of the rhetorical tradition — 



topic, comment, reinforcement % illustration, explanation, evaluation, ' 
and so on— have primarily functional definitions. We know little from 
traditional rhetoric ^about the structural realization of a category 
such as .the paragraph topic, other than the ppsitiori it typically . 
assumes in the paragraph. 4 Lacking a structural focus, the rhetorical' 
tradition has not been able to convincingly describe 'the linguistic 
characteristics of a paragraph; for instance, it is difficult to see 
what structural commonalities might unify the topic category. 

Recently, however, rhetoricians have begun to approach the 
analysis of the paragraph from the reader's point of view, a per- 
spective which, promotes questions of *a structural nature. Under- 
lying this point of view is a conception of the writing event as 
being fulfilled in a reading event and implied is an interaction 
between the writer and a reader. Young, Becker, and Pike (1970X 

instance, describe paragraphs as structure^ ^rhich "arouse and fulfill 

\ 

anticipations" (324) in the reader's mind. \ Although these authors do 
not use the terminology of readifig theory, their conception's similar 
in all important respect's to the notions encapsulated under schema 
theory (see below). Writing and reading, imply a sharing of infor- 
mation which is successfully accomplished only if the writer and 
the reader acknowledge each other's backgrounds, values, and social, 
relationships. With no basis for' sharing, no information transfer * 
will occur. 

From the other direction, some, linguists, working within the 
field'of discourse analysis are also recognizing the integrity of the 
paragraph as a grammatical unit. Longacre (1979), for instance, has 
described several. languages in which there are formal grammatical and 
lexical markers .of paragraph boundaries. He believes that these 
devices manage the integral function of maintaining the unity 4 of 
information relative to a thematic participant in narrative (and 
presumably a thematic referent in other forms of discourse). At the 
point where events change^the identity of the main participant, one 
paragraph will have been concluded and another one begun. Longacre 
continues by positing three structural parameters — concerned with • 
logical structure, cohesive relations, and stylistic factfors — whose 

/ ... • 
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various constituents interact to form a typology of paragraph genres 
and paragraph transitions. * 

Worki^flr within a recall paradigm, Chafe (1979) has also come to 
posit "the identity of the paragraph as a Structural linguistic entity. 
He had subjects vie*r a short, silent film ahd later rdport back what 
they had seen. The reports, which were analyzed for duration of 
transitions between. utterances, demonstrated that long^ pauses and 
hesitation^were reliably associated with informational "breaks in the 
coherence of space, time, characters-, events, and worlds " (180). 
Chafe ^refers to Tfhese breaks as paragraph boundaries presumably because 
at these pjftnts his subjects had concluded their information relay con- 
cerning op^ tQpic focus and tswitched to another. 

Interaction with background information 

The,*s!tecond premise of Jfhe study, that readers construct their 
compreheAsiQn of ''a passage using .their accumulated store <jf knowledge, 
has a long history %n the psychological literature. Rummelhart and 
Ortony (V)1l) take the concept back to Kant, who talked of a "productive 
imagitiatio^" characterized by an ability to apply* its categories of 
meaning to* the realization of knowledge* leant referred to the rules 
governing -this ability as schemata (singular schema ) a term which has 
been revived periodically in the psychological literature and which 
currently forms the conceptual* basis for a wide range of studies in 
reading 4n£ fcognition. As R4nelhart (1980) defines it, schemata are 
the b.uilding blocks of cognition "the fundamental elements upon which 
all information processing depends" (33). In application to reading, 
they determine the reader's goals in reading, organize the processes 
of retrieving information from memory, and guide the organizational 
path through the material. In other, related views they are cognitive 
structyre^ comprised ^f torn background information and experience 
agaihst^ which new information is compared and assessed for meaning and 
relevant . ^ . 

J e 

^^^These .conceptions Qf schemata have been called upon to expl/in 
the reading comprehension process in both its- successful and un- 
successful aspects. In successful comprehension readers are said to 
engage a pre-existing schema and use it to process the writfen 



n|aterial w before then for whatever purpose they are reading. Evidence 
of successful 'recall is mdst commonly addikc^d through demonstrated 
ability to -recall, information read or otheifwise perform in a way 
^ dictated by the written material, say in .accomplishing a procedural 
goal. Comprehension is said to fail if jfche reader is incapable of 
recalling the infofnjation. or of 'demonstrating the required level 'of 
\ proficiency. In these cases the probleci'is attributable ta some un- 

successful attempt on the reader's part to engage a schema; In the 
^jpost obvious conception of the difficulty, readers may fail to«under-» 
stand simpLy because they° lack any familiarity with the topic; they 
have no schema to engage. In more complex conceptions (Spiro 1980), - 
readers 1 failure at comprehension can be attributed to an inability . 
to( adcess a relevant schina, either for some reason buried in their f • 
motivation or approach to the situation or because the author does 
not make it sufficiently obvious what path to. take to the schema.- 
N The selection and application of a schema to a reading task » 
implies that the reader is striving to reach some learning goal/' 
Adapting the categories Uayed out* by, Spiro (1980), several outcomes ' - 
are possible.* The reader could relate the new information across a 
series of comparable past literacy events, resulting in an updating , 
of • the accessed schema. This might occur, for instance, when the* 
reader learns in. the established context of following a baseball 
team's progress that it won a game the preceding day. This is 
equivalent* to accumulating facts on a particular topic. In a second 
.outcome, the reader may expand a schema by incorporating new information, 
/or example, by* learning that whales and porpoises are mammals. This 
type of learning by itself results in additions to categorical knowl^4ge. 
.1^ however, the extended effect of the additional information about* 
whales is a reformulation of the reader's conception of mainmals 
and fishes, then the net learning result is a schema alteration. Lastly, 
in the null case, where a schema is not available, a form of learning 
\which Spiro calls compartmentalizatipn may nevertheless take place. 
Learning of this type essentially involves the introduction <>£ a new 
schema, which, being untied to any background information, probably 
has a ratjier tenuous "existence in the reader's long-term memory storage. 
Otfrer, -higher order learning effects in which schemata may be amalgamated, 

* 12 



ERIC 



hierarcta«ed^ rationalized, evaluated, and, sa on" are also possible 4 
resides of a reading event, but it is likely that they involve complex 1 , 
reflective activities in addition and perhaps occur less * immediately . 
to [the actual reading situation. - p 

In its aJcept^d sense, learning in $ school^ or testing context 
is thought to be synonymous with schema alteration and the ability to 
perform higher order interactions of schemata. When these skills 
are tested for, however, it becomes apparent that students are often 
incapable of perWrming at 'levels comparable to the expectations of 
the curriculum. Thd usual explanation is that students in this category 
are developmen tally -below their high achieving peers, the premise 
being that development precedes learning. L.S. Vygotsky (1978), 
however, has reacted to this .established notion by emphasizing the 
skills which children bring .with them to thfeir first classroom ex- 
pejrience. He sees children as possessing much of the background 
knowledge and skills necessary to perform reading tasks within their 
proper situational domain,?. Paraphrasing his position within schema 
theory, he sees children as pos^ssing m4^y well developed and 
integrated schemata which they coruld potentially apply to their class- 
room and. test-taking experiences. To explain the apparent develop- 
mental lags that tests suggest 3ome children have^ he advances the 
opinion that the instructional routine of schooling is concerned 
largely "with the assimilation ^f the fundamentals of scientific know- 
ledge" (1978:90), and not as concerned with the steps preparatory to 
assimilating those fundamentals. The usual steps to discovery, he 
emphasizes, are observation, repeated exposure and practice, con- 
versation with one's peers, and consultation with those knowledgeable 
on the topic. • In a turnaround from the pedagogically entrenched 
position' that development precedes learning, 1 he states that 
"organized learning results in mental development and sets in motion 
a variety of developmental processes that would be* impossible apart 
from learning" (1978:90). Cognitive development, in other words, 
is critically dependent on previous learning. 

In the present study we hypothesize that the structure of 
idealized reading material is organized in such a way that readers are 
le3 naturally ^through the processes which result" ultimately in a 
schema alteration. The writer in effact has made numerous decisions 
about what his intended reader knows and lays out his thoughts and 
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sentences accordingly. In the scheme of Vygot sky 1 s^ learning theory, 
each of the author's decision* has implications for guiding the reader 
froznc one stage of learning to the^next. In order to produce an 
alteration in a reader s schema, the author must gain the reader's 
attention, alert what interest; is already there, provide a con- 
vincing argument, and perhaps make the point of the argument, explicit. 
The reader, who presumably began the reading with a rudimentary schema, 
responds first by selecting the appropriate schem f a, updating it by 
acknowledging the author's topic as relevant, assimilating additional 
information to*it, and then restructuring it or not as seems 
appropriate* 

Many instructional materials and many instructional strategies 
Jail to accbmplish what they are intended to dp. But since" reading 
is an interactive process, it would be unfair to blame either the 
readfer or tjie writer, without first considering the contribution that 
each makes to comprehension; In the next section- we vill lay out a 
scheme for considering the adequacy of the written, word in conveying 

r 

a message. In a later chapter, we will reverse the focus and consider 
the reader's role in processing the message. 

Most of the previous worWin the linguistic analysis of discourse 
can be categorized into three Broad and somewhat overlapping types: 

functional analyses which begin with e. categorization of the semantic 

* v 

componentry of discourse" (Crimea 1975, Meyer 1975); propositional , 
analyses which 4 start by breaking down the logical structure of the 
sentences composing the discourse (Frederickson 1975, Kintsch 1977); 
an£ cohesion analyses which focus on the linguistic devices linking 
sentences and, propositions in the discourse (Halliday and Hasan 1976, 
Fillmore 1974). *The basis for choosing one or another of t^iese 
methodological perspectives lies fundamentally in* the underlying 

« 

belief of the particular Researcher that' language is inherently 

organized in a particular way. The propositionally-uased analyses, 

for instance, tend* to regard discourse as governed by interrelated 

♦ , 
rules, making it conformable to formal .grammatical methods of analysis. 

Out of this tradition there have been "elaborated analytical 

constructs such as story grammars (Stein and Glenn 1?78, Rumelhart; 1975); 

, #• 
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scripts (Schank V-975) ; frames (Wijiograd 1975); and so on. The intent 

\ * 

of all of these tireatmeftts has been to advance formal models of dis- 
course structure". \ Many of the researchers in this tradition,, in 

. fact, have taken tnfeir analysis as a means toward the larger goal of 
designing i intelligent • computer systems. Category-based methods are 
less concerned with yormalisms and more concerned" with describing 
the full extent of the informational complexity involved in discourse. 
The starting base for \such studies may be a comparative treatment of 
several or many languages, the intent at the initial stages being to 
contrast the structural \organizat ion of information in different 

"languages and at later sttages tp uncover the common basis of information 

^exchange across languages. 

It irf perhaps somewhat! unfair to claim that proponents of 
either the propositional or Vategory-based methods are unconcerned^ ' ^ 

v about the distinction betw^emgopd and poor discourse since the 
starting point for much of their research is how to distinguish 
random strings of sentences from true text^, And yet, their' methods. 

are basically insensitive to thia question.' Both groups, for instance, 4 

\ • 
take as- their data base actually occurriilg discourse, making no 'pre- 
scriptive 1 judjjm^nts concerning its well- formedness. The underlying* 
premise adopt^ the position of the ptoduceV of the discourse and 
states that whatever he or she regards as communicative^ is^ in fact 
communicative. .Given the premise that discourse interactively involves 
both communicants, any judgment concerning well-formedness lies with 
the /eader/hearer as well as* the writer/speaker. It is important to 
inquire into what "the reader can aijd must do with the information as 
presented in order to reach the state of understanding the writer has 
intended: *Ques*Moq^>f this sort, it would sfeem, are more immediately 
answerable by invoking methlods which concentrate <fti the- processing 
•demands of prose. These, we feef, are more direct^ addressed by 
studies focused on the cohesion relations tying linguistic units 
together. 

In this tradition, Halliday and' Hasan's (1976) senvinal work on 
semantic cohesion takes the position that a string of sentences can be 
distinguished from a true text because the text wiil exhibit a certain 
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register and a definable cohesion . JUie first characteristic places the 
discourse in an anchored setting an<h«f f ective environment, the second , 
assures that the infotmatiorial rela^ is motivated, consistent, , and 
targeted. Although Halliday anH H^skn themselves do not directly 
address the question of well-formedness, they assemble and blueprint * 
the notational machinery necessary -to discuss it — anaphora, conjunction, 
lexical synonomy, ellipsis, and substitution. 

Informational organization in discourse \ 

It is rarely if ever denied that the organization of information 
in text influences comprehension. Writing which is poorly constructed 
taxes the processing capabilities of readers. It confuses the reader . 
by 'juggling a poiAt of information in ways which make its relative* 
importance unclear ^nd its position in the discourse contrary to ex- 
pectation. And yet the widespread concern for improving the quality 
of writing, especially for children, has not yet yielded hard^vidence 
indicating how this might be done. Mdst of the current claims have been . 
m3.de by researchers working within experimental^ psychology using recall 
paradigms. The results of this research focus have almost >alw&ys 
supported a modtel of comprehension based' on a hierarchical' processing 
of discourse." Meyer (1977)', for instance, has demonstrated that 
readers selectively and preferentially recall information which an in- 9 
dependent discourse analysis would tag as critica^^o the continuity of 
the reading passage. In , other* words, readers tend to recall the 'thread 
of discourse 1 (Grimes 1975), that is, the main ideas, the topics, the 
key points, and whatever else they have been called, better than they 
do the^supporting details of the passage. 

These results ^eem ^intuitively satisfying, possibly because^ they 
confirm the impressions we receive out of out educational training' that 
a construct such as* main idea is a relatively more important component s 
of text organization -than is a supporting detail. The semantics of the 
terms themselves suggest this. And yet thf conclusions reat&ed through 
this line of research have not gone unchallenged. Spir$ .(1977) and 
Olson (1977), for instance, have argued tha^ the recall' task used in 
these experiments constitutes an unreliable measure of reading ^ < 
comprehension in that it fails to take into account the reader's background 
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or the purposes for whicH** individual readers may undertake a reading • 
task. They argue against the notion of hierarchical processing and 
for a position in which ^he reader has the flexibility to pick and 
choose from among the full range of information available within a 
passage. ' The actual choice of .material deemed important would depend 
not iso much on any 'inherent* organization of the text as it would on 
the information needs of the particular reader. "As Ols-on says, a 
reader "comes already equipped with. a very elaborate set of expectancies 
a host" of prior cues, and a *set of values as to what is worth looking 
for,.." (212): " , 

These arguments have lately been reiterated by^>ther researchers. 
Trabassd (1981), for instance-, suggests that sentences which perform * 
the main idea funct'ion in discourse are in their very structure easier 
to commit to memory than other sentences. This comment implies that 
there may fee specialized semantic 'and syntactic structures as^ociatdd 
with identifiably dif f erent ^discourse roles', an idea that is itself , 
akin to syntactic structures, such as relative clauses, direct objects, 
passive constructions,, and so on, having their own intrinsic meaning. 
It further implies that .there should be ways in tfhich to assess 
comprehension other than to use recall procedures. Jjfker an d Stein 
(1981) concur on this point,* and suggest that mors appropriate 
alternativ^f would be to assess comprehension by looking at how infor- 
mat ion is applied or integrated into other knowledge. Conventional 
reading tests do makfe these kinds of demands on readers, but the 
significance of the dejnands in particular instances is difficult to 
evaluate without a better appreciation of what processing requirements 
are entailed. i ** 

Baker and Stflin (1981> -suggest that for narrative discourse — 
material which essentially relates a set of temporally connected 
happenings or events (Labov'and Waletzlcy 1.967)-- k the story grammar model 
could provide a useful guide to assess knowledge integration, ^ Their 
claim does "not ignore the difficulties cited above regarding the use of 
formal models of discourse as psychological models of comprehension; 
it tempers them 'by suggesting they are more relevant to expository 
discourse — material which "is 'typically abstract, dealing with tin- t 
familiar concepts and situations" (41). Their claim for narrative 
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discourse is that it presents a structure which is highly determined 
through social convention, thereby . exhibiting a certain~logic , which is 
apparent even to young children. 

Categories of information in discourse 

The consensus of the literature as we have interpreted it is 
that there does exist a structured quality to discourse which guides 
the reader or listener to the intended jaeantag of the writer or speaker. 

The .structure is not fitted de novo to eacft instance of newly 
created discourse; it is to some' extent culturally agreed upon. a In 
essence this ^means that -the writer and reader share 'an understanding 
"about how the information in the full message should be organized and 
presented. The depth of the unjferstanding also seems to vary with 
the larger purpose of the discourse, the conventions being morelcertain 
for narrative than for exposition. In addition the reader musj/ have 
in mind the necessary schema to understand the content of tk^aeSsage. 
At a minimum this requires tlie reader's understanding -of the referents, 
the author takes for granted". Finally, the proper level of analysis 
is the paragraph unit, rather tfian the sentence or the discourse as a 
whole. We presume that"*Tt is at this N level where structure and function 
optimally interact. 

The given in any discourse exchange is that the originator has 

something — some refererft — in mind to say something, about . This 

* * 

referent and the^>reliminary statement made about it is»conventionally 

referred to as the topic , the topic sentence, the topic referent, or 

the paragraph topic. The topic has also been identified as the main- 

idea, although main ideas are also associated with explicit or ' 

implicit statements of the originator's in^tent. The main idea of an 

Aesop's fable, for instance, might be said to be the concluding moral. 

In a wider sense the main idea of a discourse can be any idea which 

the reader so declares it to be* based on the notion ..that readers' 

needs vary and so may not be Inet by. the' same sentence for any two 

peofl±e'.'<'< r The currency of the term main idea in the testing profession 

If * * 

seenls to reflect a pedagogical reality rather than a discourse reality. 

I The second reality of a di^coutse exchange is that tha topic will. 

be elaborated in some detail . Details logically falling within the 
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scope of the topic are of feted not necessarily just for their pure 
information value but also to implicitly communicate the writer's 
purpose.' The detail, then, more than the topic determines the type 
of discourse; at the first cut. whether It will be a narrative or 
expository account. At finer levels of analysis the exjJbsitory 
♦category can be subdivided into categories such as explanation, 
process, argument, demonstration, elucidation, and so on, and the 
narrative ^category into subtypes such as dialogue, recollection, 
and portrayal. Certain types of detail, such as description and 
characterization may equally well- serve either a narrative os 
expository purpose and muddy the border thought to distinguish the 
two sup" ^ordinate categories. 

Detail should in'its logical structured if not in its, actual 
choice of vocabulary or grammatical structure, fall within the scope 
of ^the topic. That is, the informational content of the detail 
sentences,, should legitimately qualify as information pertinent to the 
topic. Since, however,, the detail also veils the discourse purpose, 
it ofteii Happens that the originator will make this purpose explicit 
in 1 v a^ttatement following on the detail. This statement will be here 
called an assessment* - It represents rather than a necessary category 
of discourse organisation a facilitative category, especially in the 
circumstance that the reader or hearer would be reluctant or un- 
motivated to djy*w together the fqrce of the detail to make the in- 
ference-to-purpose the originator intends. 

Another facilitative discourse category is what, following 
Young, Becker, ,and Pike (1970), will be called a bridge . Its 
purpose is to inform the reader or hearer about the identity of 
the topic referent in the event that it might be an unclear concept. 

brfdge can be realized as a formal definition, a type example, 
or*even as an analogy or metaphor, any rhetorical device which puts 
the referent into understandable perspective. In conversational 
analyses it is similar in conception to try-markers (Sacks and 
Schegloff 1974), digressions from the thread of the conversation whose 
purpose is to assure that both the speaker and the hearer are on the 
same wave length. ^ . 

The type of background information provided in what 
Grimes (1975) calls the setting of the discourse is functionally 
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similar to that in the bridge, in other words it activates schema 
recognition. However, rather* than being definitional, its purpose , 
seems to be to embed the topic within a believable or plausible 
contextual situation. It also serves to delimit the range of variation 
under which the topic event could be conceptualized. Discourse setting 
is most commonly found in narrative, although certain forms of ex- 
position, such as historical accounts and' biography naturally rely on " 
it also. 

/ The, categories mentioned up to this point are those into whicfc , 
the informational content of the message is embedded. Two, additional 
categories have the purpose^not so much of Specifying the informational 
base as they do the character and reliability of the communication. 
The first of these, the register of the discourse (following Halliday 
and Hasan 1976) refftcts the originator's conception of the listening 
or reading audience. It manifests itself in the chbice of apt 
vocabulary and syntactic structures and reveals itself in an assessment 
that the style of the discourse is appropriate for the intended audience 
In this sense it is a diffuse or pervasive category, rather than otie ^ 
which shows itself primarily at the sentence level. The second 
category in this type, modality^ also tfends to be pervasive. Its * 
purpose is to declare or indicate the, certainty of the originator's 
informatiSn, for instance whether it is asserted or assumed to be true, 
whether it is conjectural or hypothetical, whether it is reportedly 
true, and so on. Both of these categories are evident in narrative 
and expository discourse, but they probably reveal their complexity 
more in, narrative. 

T^he following paragraph, which was adapted from a longer selection 
in the CAT, will illustrate the points made above. In the 'test 
version of the passage, of course, the sentences would be contiguous 

4 

and* unnumbered. • 

1. One autumn evening in the early 1800's, tttelve-year-old 
Maria Mitchell sat down and began to adjust an intricate 
mechanism known as a chronometer. * 

2. A chronometer is a delicately balanced- clock always set 
at a special time called Greenwich mean time. % 
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3. . It determines longitude and, in those days, was vital 

to ^plotting and maintaining a ship's course. 

4. The chronometer had to be accurate, and its delicate 
mechanism occasionally required precise technical adjust- 
ment. 

5. It was for this reason that Maria had been chosen for 
the job. 

" The full passage consists of four other paragraphs together 
making up af brief biography of Maria Mitchell. It is difficult to 
classify this type of discourse as either narrative or exposition, 
since in its relation of event sequences it; takes on a narrative quality 
and in its explanation of how these same events shaped the character 
it resembles an expository account. In any event the typing' of the 
passage is not particularly relevant to the ^discussion. 

^ The ( first sentence of the passage states the paragraph topic in 
the proposition 

* Maria began to adjust a chronometer. 

9 "V. -f 

It also contains setting information in the introductory, phrase and 

the descriptive adjectives. The setting restricts the rafige of 

possibilities under which Maria was acting, specifically indicating 

the event as historical, time-of-day dependent, and within the 

abilities of a child. 

The author also indicates in the first sentence that he expects 

the reader not to know the word chronometer , thereby preparing the way 

* 

for the bridge which occurs in sentences 2 and 3.< Re&ders undoubtedly 
approach a bridge expec that the definitional .terms used in it 
will be wi'thin their vocabulary range. If they are not, the purpose 
of the bridge in activating a schema will not have been entirely / 
successful. We might expect-, then, that since the autSfer indicates 
the defining term Greenwich mean time itself as requiring explanation, - 
the force' of the definition of chronometer is diilited and perhaps in- 
effectual* The purpose of the second sentence, in th^t case, may not 

'4 

be realized. 

In the fourth sentence the author provides a point of detail 
. r 

within the scope of the topic proposition in anticipation g£ "a question 

the reader might have concerning why Maria was performing in the way 
> f * 
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"she was. The point of the detail is, therefore, explanatory and in 
keeping with the overall intent of the full passage. 

The final sentence signals an assessment by the writer by making 
explicit the reason for introducing t hfe preceding detail. { The in- 
formation rather cle.arly seems to occupy a separate category from 
the detail because it conveys the< author's intepretation of the 
situation. In the historical context the information is conclusionary* 
rather than factual. The assessment also reiterates the topic and 
in this function .serves as a s link to the second paragraph of the 
passage. The second paragraph consists entirely of detail laying out 
the sequence of steps that Maria engaged' in in adjusting the chronometer. 
It f&lls, fully within, the scope op the first paragraph topic and does 
not — perhaps because of the preceding assessment-- require an explicit ' 
restatement of the topicT The reason for including the information 
within a separate paragraph probably has to do with its focu^on 
Maria -s activities, rather than on a characterization of the chronometer. 
/ The register category as indicated above is pervasive and difficult 

\^o objectify. It is realized more in the unstated contrast between 
the structural and lexical choices the author selects to cpramunicate 
a message and others that might have been selected, \ft\at we can 
loosely say is that the resulting prose takes on a- tone which is 
somewhat* aca'demic and aimed at 'a relatively adept reading group. A 
* r;ough numerical approximation to the register of this type of prose 

might be given^by applying one or more readability formulas, which' 

* 

purport tm measure the difficulty of' prose. Our own feeling, however, 
is that readability formulas have.no power to explain the structural 
basis of register distinctions, and we have not relied on them in 
this study. - * 

The modality catlfgory is also diffusely ex'^g>lified in this 
^ 7 prose passage. It again shows up more in the contrast of elements 
than in any specific structure or,worci. For example, the status of 
the informatioH presented in the bridge* sentences 2 and 3 is different 
' from tnat presented in the topic or detail sentences. „ The* change 
. in status is signaled by the use of the universal ^present and equational 
forms of the verbs ixi the bridge versus "the, simple preterite forms, in 



the topic and detail. The information *in the bridge is being declared 
true by virtue of established convention, while the topic and detail 
is declared true through an implied attestation. The author is 
silent regarding the source of the facts in question, leaving the 
reader to Jnf er tha£ that knowledge is not critical to the development 
of the account. In other types of discourse, such as accounts of *' 
scientific observations, infotmation sources might have to be carefully 
described. The conclusion stated in the author's assessment signals 
one more change of, status * this one from the preceeding attestation 
to a probable inference on the author's part. . 

Coherence and "cohesion 

At this point it is not possible to formally characterize each 
of the proposed discourse categories. More study is required before 
it can be said that they exhaust the full range of possibilities or 
that their description is adequate. Part of the justification could 
presumably be derived from studies implicating analogous, constructs 
such as rhetorical predicates (Grimes 1975) , semantic macro-structures 
(van Dijk 1977), or story grammar nodes (Mandler and Johnson 1977). 
The integration of these various concepts into a single coherent 
theoretical framework would do much to clarify each of the proposed 
discourse components. An independently motivated test of the 
significance of any set of discourse categories would be the demonstration 
that they are implicated in establishing the coherency .of discourse 
(van Dijk 1977: 16), in other words that they make more defined the in- 
tuitive judgments we have about what constitutes a well- or ill-formed 
discourse (Mandler and Johnson 1977:149). 

We will take a step to making this demonstration in the ex- 
amination of test materials iji the following chapter, but for now we 
will propose that the is^ue of well-formedness of discourse should 
be approached from two separate directions, one focusing on the 
coherence of a passage and the other on -the cohesion of a passage. 
The two terms are often taken to be synonomous, but we distinguish 
them here for the value they have in partialing out the procedures a 
reader uses in comprehending discourse. Coherence will be defined as^ 
that quality of discourse which assured that the informational base 
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is complete and intact. The presumption th^t the reading event is a^ 
communication requires that all % aspects of the message necessary for 
the reade'r to approximate the writer's meaning must be present. In 
terms of the categorical system described above, a coherent account 
would be one which establisfifes a topic, defines or. specifies referents 
so that a schema may be activated, provides information in anticipation 
of questions t$e reader may have about the topic, makes' the purpose 
of the connnunication evident, attributes information to its proper 
* source, and does all of the above in a manner appropriate to the * 
reader 's language background. A coherent account then is one which' . 
is categorically complete. 

A cohesive discourse on the other hand is one which fi conformS to 
'rule system^ for combining informational units or categories into* 
syntagmatically acceptable structures. These^systems incline the'rules 
for producing syntactically well-formed phrases and sentences and rules 
for linking sentences into well-formed paragraphs. The motivating ^ 
principles guiding the operation of fchese rule systems are still only 8 
poorly understood (Huggirfs 1977), although recent descriptive arid 
^psycholinguistic studies of cohesion relations (Halliday and flasan 
1976, Clark and Haviland 1977, Chafe 1976) t implicate memorial processing, 
as particularly important. Whether "information is new or repetitious 
in the discourse context, whether it must be held for short or long 
periods ofj^ie interaction, whether it requires assimilation with ^ 
stored information, whether it^has high or low saliency are issues whose 
resolution is described by rules of cohesion. - * . 

The cohesiveness and coherence of a passage determine both the 
availability of information and the ease with which it can be processed. 
Although each factor contributes to comprehensibility , the relative 
weight of each will vary with the individual reader. Where one reader 
might find the information and discourse 'ties sufficient to draw all 
the required inferences to activate and manipulate the intended schema, 
another reader might falter or even be misled, l*i the next chapter 
we will examine reading passages and questions in several- standardized 
reading comprehension tests with a mind to understanding the linguistic 
basis, if any, for why. test-takers answer tihe w^ they do. 
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.CHAPTER 2 
. . * INVESTIGATIVE APPROACH 

i ' ■ ^~ .. • 

The investigation was subdivided into two separate but correlated 
studies: thWfcollection of a aet of clinical interviews with children 
r whb haci taken a standardized achievement test earlier in the' school 
*year and the administration of an experimental test. The interviews 
were designed, to probe children* s awareness of those points in 
individual test reading passages which we had hypothesized would 
interfere with the formation of a coherent and cohesive account . The 



refculteq^f the interview investigation were expected to feed into the 

? * . t ixperim^nfeal test l>y suggesting particular ways in which test passages 

Qould-'be rewritten to increase comprehensibility. Any positive effects 

would be ascertained through enhanced-performance on rewritpwf'i terns. 

.Since a' large constellation of subject and test variables enter * . 

. \ixttQ the determination of performance we felt it better to maximize 
.* r * <^ 

o„ur exposure to many of these variables in the interests of tagging those 

, which^are relatively' more important and eliminating those which were 

-inconsequential. Subject selection and investigative procedures, then, . • 

were Resigned to expose us to a wide mix, of variables at the expense 

of including a latfge number of subjects in any one cell. It was ******* 

expected that teats of statistical significance would have to be *c r on- . ' 

feetvative for this -reason. 

♦ - Subject selection m 

Subjects were* drawn from the third, fifth-, and eighth grades of 

the Montgomery County (Maryland) Public Schools and from two proficiency 

levels within each grade. The students wer§ selected from these \ 

grades in oe thefchool district administers the California Achievement 

-Test {CAT) at these levels. Because it would have proven difficult to 

* «* -draw students at both proficiency levels from the same school 

especially students in the two lower grades, the students were drawn 

/ ' from fbur different schools: the 3rd and 5tli graders from two * * — - 

% " > elementary schools, and the 8th graders from a junior hfgh school. The 

neighborhoods of t^ie four schools were not matched sodlo-economically 

* < „ 

«and there is good reason to suspect correlation between the proficiency 
♦ levels of the students and the economifc and educational- standing of 

" their,, parents. All four of the schools* however, were participating in 
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m thq. same experlniental reading curriculum so that there was at Xeast 

^» * 
4 a degreja, of uniformity in their reading instruction. 

The proficiency levels were defined as high and low although' 

in. fact the low group was actually perfojpiing at close to the national * 

nqjrms on the CAT. The initial determination of proficiency was made ^ 

Dn the basis of the student? 1 scores on the CAT administered in the 

r r 

Fall of the school year. Studentswho scored at the 7th, 8th, or 9th 

stanines on the CAT's reading comprehension subtest were considered 

• *• . ..high achievers, those who scored at the 4th, 5th, or 6th staines were 

considered low achievers^ Within the lower achieving groups we 

imposed a second selection criterion, namely that students 1 scores on 

the vocabulary subtest of the CAL be as high or higher than their 

reading comprehension • scores. We hoped in this way to eliminate from 

<Stm£ideration children whose l^ck of reading proficiency might be 

attributable to decoding problems. As a final consideration teachers 

or administrators familiar with the students were consulted for their , 

impressions of the. correctness of a designation. In a few instances 

they, felt that the CAT test scores were unreliable for a particular 

student, in which case the student was dropped frorn^ consideration. 

' The parents of each potential subject on the approved list werfe 

then contacted by mail to -elicit approval fQr their child* s participation 

in the study. It was explained that the chil4 should be appraised of 

the study and, his or her willingness tp participate determined. ^ * 

Approximately 70% of the parents responded positively to the request 

with the majority of the rest not responding at all. Only a few 

fc *, * .parents voiced their unwillingness tor ]^t their child participate. 

* . f Since most of the non-responding parents were .among those of the low 

achieving students* an attempt was made to contact all of these by 

ph^ne to give them a more personalized account of the study. This 

resulted in several more of them granting permission. * 

We had expected to use 15 students at each of the three grades 

and in each of the proficiency groups for a total of 90 students. 

'In all but the case of the high achieving 5th graders we were able 

to reach our required number. For the high 5th graders we made up 

* t^he, difference by drawing the extra students (two) from the school . , 

, ,whete the low achieving -students were selected. This was felt 
- * * * »„• ' * */ * * * * • * 4 

justifiable since no control over situational or environmental 

variables, wap attempted. In fact, the students so selected did not 

vafry in any discernible way from their peers'in the other school. 
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All of the students in the final roster were white, were native 
English speakers (two were bilingual), and were in the appropriate dge 
range for their grade level. All of them had been characterized by 
their teachers as presenting no significant motivation or attention 
problems. 

Interview procedures 

Of the fifteen or more students in each grade-proficiency group, 

five were randomly selected to be subjects for the interviews and the 

.remainder subjects for the experimental test administration. The 

only control consideration was to balance the selection of subjects as 

evenly as possible between boys and girls. The interviews were con- 

ducted individually with the author and an ^assistant. At the sjzart of 

each interview the students were informed about the study and encouraged 

to answer questions openly and without anxiety. They were specifically 

told that they were not being evaluated and that the questioning w^s 

for <he purpose of determining whether improvement in the tests was 

warranted. They were encouraged to examine the tests critically and to 
■ * • * * 

reflect on their answer choices. # 

Each interview lasted approximately one hour during which fo\ir 

or five passages from the appropriate grade level of the CAT (Forms 

C and D) were presented for comment* Sqpe of the passages had been 

'included on the form of the CAT the students had taken in the preceding 

fall, * but few s#bdents actually- recalled any of the passages and fewer 

yet indic^ted^ that they remembered specif 1q questions. Each reading 

passage was typed individually on*a single sheet of paper. The 

student read each passage silently after which the original questions 

asked of the passage were presented one by ?ne on individual j^x 5 

cards. After each answer, the Interviewers directed a series of * 

probes at the students, the purpose o^whictvwas to discover the 

assurance the student felt in answering the question, the reason for 

answering incorrectly if this were the case, and the possibility the 

t f 

student could be directed td see the applicability of another answer. 

f 

All interviews were recorded and later transcribed. 

The interviews were conducted in 'a clinical manner, meaning that . 
each question on our part was motivated by the particular situation. 
No attempt was made to standardize the questioning, other than to 
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assure that the student was looking at each passage question critically. 
Preparatery to actually interviewing, however, a number of problem types 
had been isolated in the independent analysis <?f the test passages 
and questions. Each of ^hese suggested a particular line of questioning, 
the thrust of which is suggested by the probes listed under each type. 
In the examples illustrating each type, the expected answer is starred. 
» 1. Plausible dis tractors . The organization of the passage 

suggests that one or more distractors is plausible either because -they 
are coordinate at some level of structural organization or because one 
is logically included within the scope of the other. 

Example: Balloons (CAT 15C) 

The passage' suggests that the first piloted balloons 

were use3 primarily for * * 

*a. sport 

/ h. warfare 

c. instrument testing 

» d. weather forcastingS — — 

# Both a and b are probable answers, since they are coordinate withia r-— ' 

j \ 

the^topic continuity: a is cued by the- dates 1783 and 1785 and b by 
the information "before 1800". The diffij^Lty comes with the word 
4 "primarily" -which requires an interpretation o J n the reader f s part. 

* Example: Ruth (CAT 13C) 

\ 

Why was the copy of the map made out of metal? 

*a. so that it would last a long time 

b. so that it would fit inside a school 

c. .so, that many people could work on it 

d. so that the children cou^d take It home m 

1 C is a plausible alternative because the passage states that the map 
is outdoors for people to enjoy. Under an interpretation t,hat th^ 
enjoying involves hands oil manipulation, then both £ and a are logically 
relatable. 
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Probes : How sure are you about your answer? Is there 
another answer that might work here almost as well? * 
Do you see *ny connection between answers £ and £? If 
« ' someone chose answer x instead, how would you convince 
him that your answer was better? 

• 2. Difficult inferences * The question asks for causal infor- 
mation leading to or motivati^ a targeted event,, but the information 
in the passage is laid out in such a way that the motivation can only 
be inferred from the events following from the targeted event. In 
'a variant form the question requires an inference, for which no infor- 
mation is explicitly stated in the passage. In both closes the reader's*' 
ctations about where the information may be located are not realized* 

Example: Sequoya (CAT 15C) 

Why -did the Cherokete nat^tn call a council? 



k€fe nat^tn 
.^^ider 



a. to name a new 

b. to decide where to move 

*c. to discuss Sequoya's alphabet 

d.* to meet % with tfte white settlers 

Information pertinent to answering the question unexpectedly -follows 
rather than precedes mention of the target event.' 

Exprfplei Maria (CAT 18C) 

The Mitchells sent the record of * their comet observations to 

Professor Bond because they wanted him to 

1 _ _ . _ * • 

a. name the comet 

b. complete the^r calculations of the comet 

c. tell* King Frederick* VI about the comet 

*d. confirm that no one else had discovered the comet 

The -passage mentions only that Professor Bond sent the Mitchells good 
news, making it conjectural what his role was. "it could be assumed 
that his role corresponded to the statements in either b and d. ' 
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Probes : How sure are you about ^our answer? In what par* 
of the story can you find your answer? Do you have to make a 
connection between different parts of the story to answer; 
the question? Some kids say that £ is the fight answer. Do 
you see how they might come to answer that? Do_you think 
that the story is leaving something important out? What should 
it tell you In order to answer jr? 

3. Unstated point of view * The question is unclear on whether 
it wants the test-taker to answer from the point of view of the pro- 
tagonist or the point of view of the objective obsewer. 9 i 



Example: Andrew (CAT 15C) 
v 

Which of the following is probably the maifo reason Andrew 
Warrel likes the' art show so much? 

r. 

a. He likes animals . 

b. He writes for the school newspaper. 1 
*c. He is a member of the Pen and Ink Club. 

d., He is a student at Oliver School. 

Andrew Warrel hlsnself is presenting t evidence to point to <i as the correct 
answer, while an objective analysis would also reveal that he is a 
member of the club sponsoring the show. The passage itself is presented 
as a letter with the^ mention of Warrel's club membership given not in 
the body itself butj in the title following his signature. 

Example:* Markgon (CAT 18C) 

Which of the following best describes the surprises 
the package being advertised? t 

a. magical - , * . . 

*b. unknown 

♦ 

c» healthy 
d« delightful 

From the salesperson's point of view the answer he would like the reader 
to select is either ji or d, whereas an objective assessment of the 
claims he is making would lead the reader to Select b^. 
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Probes: Are* you sure about your answer? Did you consider 
dny other answer's before you chose your own answer? Would 
you answer any differently if you didn f t believe (the pro- 
tagonist)? 0 How would you decide if x or -jr is the best 
answer? How wduld you convince someone that your answer, 
is the best one? 



4. Hidden detail , The needed point of information is locatfed 
within a discourse catego*y_where the reader would not ordinarily 
expect it. , 

Example: Sequoya t3 (CAT 15 C) 

Sequoya fpund that his alphabet 

*a. used eighty^-six signs 

b. took six years to finish 

c. would not h,elp his people 

d. could not relate to the spoken langtiage * 

The information asked for^is included as a detail within a formal 
assessment of the paragraph topic. It is consequently completely 
incidental to the function of the sentence in which it is located. 

Probes:; Is this information important to the story? Can 
you point out whete your answer is? Is it hard to find where 
the answer is? Where did you expect the answer to be? 

5. Reliance on special iced background information . The test 
question requires 'the reader to bring information to bear which is* 
tangential or irrelevant to the story. 

Example: Balloogg (CAT 15C) 

According to the passage, what effect have scientif ic^discoveries 
had on ballooning? ^ * * 

«, *a. Ballooning is , safer. 

b. Ballooning 1 is*more expensive. 
% . c. Ballooning research is restricted. 

d. # Balloon steering is more accurate. 

y 

Readers must understand that individual events mentioned in the story, 
such as the discovery of helium, constitute a scientific discovery. 
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Probes : How sura 3 are you about your answer? How did you figure 
®v * out the answer? Was there enough information in the; passage 
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to answer the question? What do they mean when they ask 
about (the point of background information).? Do you think 
this is a fair question? ' 

6 * - Inadequate identific ation of referents . The phrasing of .the 
question leads the reader to expect that a referent has been explicitly 
identified when, in fact, the passage* treats the referent as undetermined 
Example: Andrew (Cat 15C) 
I" Which of the following is a fact about "special prize"? 

. a. Andrew Warrel will receive the -jsrize for' writing a 
good letter. 

b. Oliver School won the prize for their decorated balls. 
*c. Andrew Warrel wants the club to receive the prize, 
d^ Some of the drawings won the prize. 

The question misleads the reader into thinking some specific prize has 
been discussed in the passage. / 

Example: RutlT (CAT 13C) * 

What did the girl use to get paste out of the bowl? 

4 4 
a. a stick J * 

*b. a knife , 

c. a brick 9 . 

^d. a paper j 

The question directs the reader to'look for some specifically mentioned 
girl in the story. * , 

Probes : How easy was it to pick the answer? Did you have * 
any trouble figuring out who they were talking about? What 
can you tell me^>out (the inexplicit referent)? Is (the 
the inexplicit Mferent). an important part of the story? 

7. Vaguely worded questions . The phrasing of the question 
itself leaves the reader in some doubt about the test-writer's Intended 
meaning. 

Example: Alley (CAT 16C) 

Which of the following phrases dqg£ribes the alley as 
though it relates to a person? 
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a. "of an old piano" 4 ■ r><f 

b. "in a far-off^ ungle" 

*c. "in the back pocket of the city" 

d. "on. the broad downtown streets" 

The verb "relates to" is uninformative in specifying the type of relation- 
ship* asked for* 

Example: Ruth (CAT 13C) 

Which word best tells about the boy? 

a* lazy. 

b. silly ^ ' 

c . brave 
*d. helpful 

The question is stated so broadly that it leaves open the ^possibility 
that any of the answers might b e e justifiable. ^ 

Probes: How sure are yo.u about your answer? Do you find the 

4 

question easy or hard to understand? Is there another answer 
that might work almost as well? How would you change the 
t question so that someone would find it easier to answer? 

8. Absence of gjuperordinate topic . The question suggests a relation 
between two referents and implies that the relation is mapped in the 
passage by a statement of topic force which subsumes both; however, the 
necessary, statement is not explicitly stated. 

Example: '^ Guitars (CAT 18C) 

Today 1 s guitar is < most liVe the guitars made by # 

, a.* Zyriab * r 

b. * August Otto 

c. Hector Berlioz .-»©' 
*d. Antonio Torres " > , 

The reader must align passage information within the scope of two 
separate topics in o^der to answer the question. The cohesive link of 
each of these topics" to a logically superordinate topic is missing* 

Example: Cosby (CAT 13D) 
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The story says that Bill Cosby has been on TV shows to 

a. play football 4 

b. make records 
*c. ^muse people 

d. race go-carts 

The question asks for topic level information which the passage itself 
does not provide. In this particular case the lapse is especially 
critical because the question can be interpreted^Jo mean either 'why 
has he been on TV 1 or 'what has he done on TV' . 

Probes : How hard . was it to answer this question? How did 
v you figure it out? What kind of connections do you have to 
make to answer the question? Is all the information where 
you expected to find it? 

♦ 9.' Passage independent questions. The question can be reliably 
answered without reading the passage. 

Example: Andrew (CAT 15£) ' 

Which of the following phrases is an opinion expressed in 
the letter? 

a. "I am writing. . ." 

b. '""It will be shown.. ." 

c. "It's the best show. . ." 

d. "Ttiree are ink drawings.*." 

Assuming that a reader has a clear idea of what an opinion is, the 
question can be answered without consulting the passage* - « 

Probes : How easy was it to answer the question? Can you*find 
the answer in a particular part of the story? If that patt 
of the story was missing do' you think that you could still 
answer the question?. Why do you think that. this question was 
*o easy? 

10. No best answer. The information in the passage suggests as 
the most plausible answer an option which is not included in the question. 

" Example: Sequoya (CAT 15C) 

Which of the following best describes Sequoya? 

% 
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a. lazy rf " J 4 

b. lucky 

l *c. selfish , 

*d. intelligent > 

The passage provides no direct infotmatibn which would support one of 
these answers oyer another. The best answer, which would be persistent 
or concerned , is not included. ^. * 

> 

Probes : How satisfied are yoy about the choice you made? Do 
you think there is some other choice which should have been 
included? If someone said that none of the choices was good 

4 

what would* you say? 

Following the substantive part of the interviews, and depending 
on available time, students were questioned .about their feelings regarding 
testing, their attitudes toward reading, and their sense of how their 
reading behavior shaped itself to the situation of test taking. 

All of the students without exception were cooperative during 
the interviews, although they spread themselves over a wide range of 
variability on their willingness to initiate discussion, to volunteer 
information, and to test various options. The analysis of the inter- 
views Is presented in the following chapter. 
Experimental procedures 

The experimental procedure was designed to test the hypothesis 
that reading passages from three standardized reading comprehension , 
tests — The Iowa Test of Basic Skills, the California Achievement 
Test, and the Metropolitan Achievement^Fest could be rewritten, such 
that they were more easily comprehensible. The measure of comprehension 
was performance on the^game set of questions which test-takers were 
required to answer on the original \tersions. 

The complex considerations which entered into making individual 
changes are more fully described in Chapter 4. At this point it will + 
suffice to say that the rewriting resulted in >no significant changes 
in the determined readability of any of the passages, that no new A 
information was deleted. Essentially all changes were made in consideration 
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of the theory of .discourse structure laifr out in the preceding chapter, 
dottier than, to attempt a consi^tenc^in the formatting of passage ♦ 
information, no other control over N thd type of change mad^Was possible " 
since each passage selected for rewriting presented unique pre 




of discourse organization. Furthermore, the aggregate of tne passages 
selected for any one grade level demonstrated a wide variety of 
discQurse types within the broad category of expository prose. The 
purpose w^s to select items which £n their totality represented tjje 
full range of stated test objectives. * - * " . 

f " Individual t^est pickets were constructed for eaqh student' and* the 
packets were mixed for original and rewritten passages. Third grade 
students rea'd four passages altogether, 5th graders and 8th graders 
each five passages. Students, of course, were not, inform^ abou* whether 
a particular passage they were reading was ariginal or rewritten. Each • 
of the packets was designed to be easily -^j^Leted within 45 minutes, 
and,- in fact,' fio student was unable to s complete the test in the allotted 
time. I * ■ ' 7 ^ • _.- p * 

The tes\ was in most ( respects administered under conditions 
similar to tho^e^j&Hrfcfr would have obtained under -a real testing, except 
that a machine scorab'le form was not used. Students were, instead, » 
asked to indicate their choice in an appropriate blank. In addition, 
they were asked, for each question, to provide a confidence rating 

f 

for 4 their selection. The point of asking for this information was 
to focus the student t s attention to the task, not to provide another 
scoring variable*. Consequently, except as it was useful in disallowing 
the data from one student, the confidence ratings are not further 
discussed. s ' 

A total of 63 students were acjministfired the experimental <tg£t 
with the breakdown by grad^,and prof icienty level f as indicated in * 
Figure 2-1. . H " % 
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Figure 2-1 

Stlbjecft> population on experimental test & 



3rd 
5th 
8th 



lowj Achieving 

* 11 
10 

11 



High Achieving 

n 

10 



In addition one other low achieving 3rd\rader took the test but his 
^scores were .not tabulated, sipce there^was reason to believe he' guessed 
at all answers. . 

Following the test administration we attempted some informal 
interviews with students in group situations to elicit their feelings* 
about the relative difficulty of individual items. It was hoped that 
students' opinions might systematically vary as a condition of which 
version of a passage was read. Unfortunately, time constraints and 
logistic' difficulties made it impossible to approach the end of exam 
interviews with any precision and they are not further considered. 

4 
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CHAPTER 3 
INTERVIEWS 

' The interview procedure was initially conceptualized as a fact- 
finding study to identify the "characteristics of reading comprehension 
.tests which perplex children at various ages. It was expected that 

r 

r 

the types of structural problems we had- identified ^in the tests would 
likely prove difficult to children, but that they would be able to 
vocalize their difficulties and suggest, improvements^/- -We then expected. 
. to implement their suggestions in designing the rewrites of individual % 
test passages, /and to verify the effect of the characteristics in an 
experimental test. % * 

What In fact resulted was much less straightforward. Students 
as a whole did find-certain of the' hypothesized problem types difficult, 
but others they found to be relatively untroublesome. With certain of 
the difficult types, the subjects sorted themselves «by proficiency 
groups, with the lower achieving subjects experiencing essentially 
all the problems.' With Certain other types, however, students at both # 
proficiency levels professed difficulty, but the er achieving 
students produced proportionately fewer errors. This skewed pattern 
of results suggests that the ten identified problem types may not be 
equatable on the sam J parameters and that a finer analysis is necessary 
in order to understand the, mechanisms underlying judgments of difficulty 
and poor test performance. We will suggest in the following discussion 
that factors other than those which fall under the usual set of reading 
comprehension objectives are in part responsible for perceived and] 
actual item difficulty. The results will show that difficulty is often 
attributable to not knowing the appropriate ^procedure for verifying 
an answer <Tr for selecting* a comprehension strategy. These factors in 
turn may be conseqtlences of a student's negative attitude toward tests 
or to his own test-taking skills. The distinctions between each of thesi 
factors and the skills 'encompassed in ff p(ire comprehension**"' are weak, ( . 
allowing for some conceptual overlap. In the following discussion, 
then, we will not attempt to rigidly define each of -these categories 
other th^n to characterize them as metacompr^hension and test-wiseness 
skills. 
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'The claim that we will make from the interview results lis that 

* * 

reading tests, in addition to their explicit purpose of assessing 
comprehensionrskills, also covertly assess metacomprehensioir and 
test-taking skills.* To the extfent that test developers do ^ot recognize 
this point or are^^^horaik of the interrelations between these various 
aspects, the breakdown dx skills which they claim a particular item 
measures is likely to be erroneous or only ^partially correct. It would 
be simplistic tociSm, then, that a low-scoring student may need re- 
mediation In some comprehension skill, such as drawing inferences, 
without first ascertaining that the fault does not lie in some other 
direction. In the discussion we will concentrate first on detailing 
the variety of extra-comprehension factors wt\ich influence test per- 
f onpance and later- consider the effect that poorly written* passages and 
test questions Cdn^have -on d£f lifting readers. 

^spxehension Strategies 
^ * " J- \ ** r " 

^Qu^^early^into the interviews it became apparent that both high 

ant*£ow acfifceving students were operating with the same repertoire of 

colro^hensiOT^kiils^ if we restrict the t^rm.to'the usual array of 

ljjfer al , J $ftt jb^entiai , and evaluative, skills. The high achievers, 

^)weyer,*^ere more successful in consist^itly applying these skills' £o 

the solutituv off particular problems^ while the low achievers almost 

inexplicably would one time apply a needecfr skill and another time not. 

lat wa# particularly tro\*bling was that an individual student's 

performance* could not be predicted for any particular item... Excep^^ 

for\a few items with inherent' difficulties in the ir{^cons true t ion wMh 

pcyed problems JEb? air students., no ixem initially judged to be without 

structural inaccuracies was, i^*ari4bly .failed. Contrary to what one 

tolght pifes^pt abo^it a comprehension sTci lis -gap between high and low 

achieving students, there appeared to be on^y a difference t in the 

frequency with which particular skills were applied/ 

Questioning,' fiowe>)tr, dic^ eventually make it clear that 'an 

absolute dif f ectfffce could \& found $o explain the scatter in the 

response pattern^ Rather than involving a traditional comprehension 

skill, though, it involved Vhe selection of a particular processing 

routine:* either ^fiierarcftical processing, strategy, which represents 
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the preferred algorithm for reaching the correct .answer , or what w^. 
will call a detail processing strategy, in which all information is 
essentially. equated for relevance. It is probably the case that the 
two processings models can operate in tandem or that one call supplant 
the other for some period during the reading event, but' in general, for 
aay particular passage, students seem to prefer one to the other. 

The distinction is clarified in the fallowing two exchanges, each 
dealing with the same question on the GuitA passage (CAT 18C) . The 
.question read? e t 

Which of the following is a guitar-like instrument 
from Asia Minor? * 

a. lyre 

b. nefer * ^V^- 

c. cithara 4 
■d. # vihuela 

The passage itself mainly concerns the* history of the guitar, and as 
, the- fallowing segment shows the information relevant to answering the 

question is incidental to the theme. , 

One of the earliest instruments shaped like today's 
guitar yas thfe nefer of ancient Egypt. Later, other 
'instruments related to the guitar were developed. Among 
these werfe the vihuela from Spain and the cithara from 
Asia Minor. * 

The first student is a low achieving 8th grade female reader (CK) *who 
nevertheless successfully answered most of the questions on*the passage, 
including this question. The interview, begins by asking for her opinion 
of the importance of this question. (The letter I indicates the 
interviewer and th£ letter S the strident subject.) 

I: Do you tjjink what they're asking you for is an 

important part of the story? ^ 

S: No. 

I: If. it's not an important part of the stocy, how did you 

know where to find it? 
S: It'd probably be in the middle. 
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I: And how would you go about finding it, if you 

didnlt remember it from reading the story through the 
fj.rst time? 

S: Look through it real quick for the word, 
. I: Which word would you look for? 

S: Look f9r the first word (i.e., the first option).- 
' I: Do you know any of these other words (i.e., the 

other options)? Have you ever heard of them before? 
S: I've heard of that* one (caption, d) I It's a musical 
instrument. (Later * it became* evident that she} had 
understood vlhuela to mean viola. ) 9 
^ I : Did you* think this question was fair? 
S: Yeah. 

Later in the session and relative to another .que§t ion she clarified 
her conception 1 of fairness. , 

■\ - 

I: I want you to think about whether or not this is a 
j fair question. 

S: No, because they don't say it in the story. .. .they 1 

make a hint, but it doesn't match up with any of them <• 
(i.e., the answer options). c 

It Is clear ,from the larger context that. she means none of the answer 
•options is clearly indicated in the passage; either overtly or through 
a meaning link. These excerpts indicate a preference for the detail 
processing strategy. Information is evaluated not so much for the role 
it plays in a paragraph; but for its own sake. 

In the second dialogue a high achieving 8th grade male (3A) 
implicates quite a different processing strategy , % one in which hier- 
archical relations become heuristically important. # 

I: Have you ever heard of these instruments before? 
S: Yeah, I've heard of the lyre. 

I: The other three possibilities (i.e., the other answer 
options)? 

S: No"; I haven't heard of any other than the lyre. 
I: 4 Does that at all jflake it difficult to locate something • 
if you don't know what it is? - 
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S: Not really, because it says in the passage that 
they're guitar-1 Ace instruments and I don't 
expect to know all the instruments. 

I: Just the very fact -that it's a word that you've never 

seen hefore, does that make it any more difficult for 

' * you? • r 

» * % j 

, S: No, that doesn't make it any more difficult ....You / 

can tell they're all musical instruments. You don't 

need to really know the definition.^ 
I: Is this question asking for an -important piece of 

information?, * 1 

S: No, I don't think it's really that important. It's 

critical reading I guess, to pick that out. Knowing 

that the cithara came from Asia Minor is not what 

they're trying to get across. 

Notice that both students have accurately assessed the relative 
importance of information #sked for, indicating that each has a clear 
conception of the passage theme arid probably also the organization of 
discourse categories in the paragraph. Each also agrees ^on the merit 
of the question although the high, achieving, student mislabels the 
demand as "critical reading. " The difference between the two- lies in 
the conception each has of the information per se, the lpw achiever 
seeing it as coequal to other passage information, the achiever 
as nested within a more encompassing detail. As we have seen in this 
example, it is a difference which may not reveal itself in a difference 
in response, but only in a difference in .the procedure used to Verify 
the selected answer. 

Each student found it necessary to recheck the passage before 
answering, but each went about it in a different way. The low — 
achieving student systematically scanned for each of the alternatives, 
while tlje high achieving student presumably first located the super- 
ordinate concept from his memory of the organizational hierarchy and 
then verified # his choice. Both procedures involve a process of 
elimination, but the process operates at two different levels of ar^lysis, 
the one making no assumptions about the location or the connectedness 



ERLC 



42 



* 4 



of the options, the other making an assumption that the options are 

m r *; 

associated semantically in an inclusion relation and* that they will 

most likely be found in an. equivalent location and in the same discourse 
function within a paragraph. 

Each of. oar assertions^ . of course, is conjectural, but" each 
holds v up through repeated instances of asking students to locate in- 
formation. .The detail processing strategy reveals itself in scanning 
behavior; the hierarchical processing strategy reveals itself in a 
directed search to verify some hypothesis. Either approach can be 
successful and either* can lead to an incorrect choice, but overall, 
the, hierarchical processing strategy leads to more correct answers, 
if only because cert ain^quest ions predispose thtemselves to this kind 
of analysis and because it saves' time. It is, however, a strategy 
which many low achieving readers prefer, to avoid probably because 

it reduces the certainty of response. 'Many of these students know 

* 

that detail processing will be successful in instances where the 
information logically falls within discourse detail ^..Jlierar chic al 
processing risks seeing this information as trivial and relegating it 
no status in memory. If the passage information should be queried, as , 
it often is in comprehension tests, hierarchizing readers may not 
have access to it. The detail ^processing strategy at least assures 
those who select itvthat _they will answer correctly as many questions 
as they expect to answer beforehand. As the low < achieving student 
above put it in response to another question 

I know I'm goiAg to get at least one wrong. 

Outcomes- of Detail Processing Strategies 

A reader's decision to rely on a detail processing strategy 
should produce some success with the types of comprehension questions 
which require a direct or easily inferrable match-up between passage 
content and answer option, but it can lead to difficulties in processing 
questions which depend on an Awareness of the hierarchical relationships 
between information units or which require a tracking of information* 
flow. In \he first case it may happen that the reader fails 1 to 
apprehend the main idea of a passage or, more' commonly , loses sight 
of the" relative importance pf information. For. instance, a low 
achieving 5th grade female student (MY) responded in the following way 
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to inquiry about the following question from the Balloon passage (CAT 15C) 

In what year did Letitia Sage fly in a balloon? 

The relevant paSs&ger information is ^contained In a discourse detail. 

* # 

Among the first adventurers to try ballooning was Letitia 

Sage, who flew in a balloon in England in 178f5. 

The student was not being successful in locating the information and 
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after some time the interviewer began the dialogue. 

I: Do you think that what they're asking for is 

important to the story? 
* S: Yes. m N 

I: How do you mean that it's important to the story? 
> S: Because right now 'we're working on reading cbmprehension. 

And would they know that you really read the story? 
I* t.O.K. , so it's important if somebody's going to ask you 

about it. Is it important if you were just going to m 

read it for your own pleasure? Would you remember ' that 

sort of thing? , 
S: I would tend to remember it. In case my^nom starts 

asking me questions about the. story; #fcu know, did I 

have any problems with the story, and stuff like that. 

So then I get .teaching at school and at home, 'cause 

of my mom and dad. 

The spudent is evidently 'basing her selection of a comprehension strategy 
on the importance that others attach to content, not the importance 
that the*information might take given a logical analysis or a highly 
felt personal motivation/ She is, in. fact, riot reading for herself; 
she is reading for someone else. , 

, Tracking difficulties are particularly evident among the younger 
students. For example, a 3rd grade low achieving female (AC) correctly 
answered the following question «#rom. the Cosby passage (CAT 13D) with 
th* fourth alternative 
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WJien he was a boy, #111 Cosby 

a. Made records 

. b. became famous *i 

d. appeared on TV \ 

d. made people l&igh 

but through a more involvgd prffcedure than the text warranted. The 
relevant -part- of the passage read 

Can you tell stories that make people laugh? Even 



vhen~he ( va8 a boy, Bill Cosby was able to do this. / 
Hid teachers and friends enjoyed his storied. 



As is almost obvious to an adult reader, the answer is found by making 
aii inference between the first and. second sentences. The young reader 
in questiBh, howeVer, failed to spot the connectedness of this Infor- 
mation and opted for a processing routine in wh,icfi the deictic word 
this ending the second sentence was interpreted as pointing in a 
forward rather than backward direction* , This wadwrevealed in the 
dialogue, ^ * 

I: Are yot/havi^ng trouble (answering the question)? 
S: 4 I can/t decide. 

I: Which) ones are you thinking about? 

S: 1 Made people laugh' because her 7 is funny no*r,~ so he 

had! to be funny when he was little. * 
I: Is tnfere any part of the story wfcere it actually told 

you he made people laugh when he was a boy? 
S: No. 

< 

I: It doesn't say that? 

S: No. It says 'Can you tell stories that make people laugh? 
Even when he was a boy, Bill Cosby was able to do this. 
* His teachers and friends enjoyed hi§* stories. 1 So he's 
got to be f\mny if they enjoyed his stories. 
I: Oh, I see. What does 'this' mean? (Interviewer pointed 

to the occurrence of the word in the second sentence,) 
S: It means he yas able to make people laugh. 
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I: When you see this word •this 1 does it mean what's goiA'g 
" ' to fol^qw it? , 

s! Like, Bill Cosby^was able to do this :> his teachers... 
(Emphasis in the original.) 

We would contend th^t the child is not suffering from some inability 
to m§ke inferences, since she does successfully infer an answer from 
the evidence* in the third sentence. She is processipg material in 
each sentence as if it were essentially independent except when a 
specific cue like the deictic word indicates an explicit connection. 

The same student gives what may be another example of the effect 
a detail processing strategy can have on comprehension. In another 
question from the same passage the reader is asked v t 

* The story says that Bill Cosby hats been on TV shows to 

a. 1 play football p - 

b. /4nake records 
e. amuse people 
d. race go-carts 



The relevant passage information is r spread over two paragraphs. 




His records and TV shows have made him one of the best- 
liked and most famous people of his' time. 

1 In^his shows, Bill Cosby often talks about the things he 

.did when he was a boy 4 He tells stories about playing 

football in the street and racing go-carts. , % 

The student 1 selected the fourth option because 

, S: ...I remember they said in the story •go-carts 1 some- 
where, or other* 

r 

The dialogue continues ; 

I: G-K., but here they say about playing football, too, 
don't they? Maybe that's the right -answer then? 

S: O.K., let's see: 'playing football in the street 
and racing go-carts 1 . 
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I: Did he race go-carts when he was on TV? 1 
S: No, it doesn't say that. ^ 

I: Would you still answer it that way, do you think? 

Even though it didn't, say that? 
S: No. I don't know. 

I: Does tjie question ask you what he d'iii on TV 

or why he went on TV? 
. S: The first one: What he did on TV. 
I: Does it talk about being on TV at all in the story? 
S: It says 'His records and TV shows have made him 

one of the best liked and most famousjpeople of his 

time. 

I: O.K., dogs that tell you why he went on TV? 
S: No. 

I: So we're back to the. start then. How' re you going 

to answer this question? 
S: I have no idea. 

What has evidently happened in this student's reading of the 
question is that she assumes it to refer t<^ an explicit activity 
which Bill Cosby performs, on TV, when in fact the intent of the quest;Lc 
is to determine the effect of his performances The question itself 
permits either interpretation, but tnje second one is almost precluded 
if the processing strategy does not ajllow for a linkage between the 
previous mention of making people laugh and telling stories on TV. 
Although we did not extend the interview, we would suspect that the 
child would have preferred and immediately chosett &n option which read 
to 'tell, stories.' To her mind, then, there was probably no correct/ 
answer among the options. Also, as the questioning shows, she had no 
appreciable difficulty in comprehending the passage using her set of 
preferred procedures. 

Verif icational Routines * . 

The interviews also 'revealed a significant difference between 
high and low achieving students* in the procedures they use^ to verify 
an answer once one had been selected. In general the high achieving' 
f students were more concerned about verifying information, and they 
set about it in a mpfce principled way: for unambiguous questions 
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simply finding as close an equivalence between passage content and item 
query as possible and for ambiguous questions adopting a process of " 
elimination procedure. For instance, in the following comment a, 5th 
--^•^ grade high achiever (CM) describes her preferred routine. 

— ^* • See, lots of times in th^Se questions that they 

have on the test, they don't say which one is not 

this. Some of the questions you have to look through 

* 

and find the one that is most reasonable. Lots of 
times, none of them are, but ypu have to just look 
really closely. t ' 

An 8th grader (SG) proffers a similar opinion in dialogue regarding 

an item that sjie answered assuredly and without error. 

. " ' ' ' . : , ' ' • 

p I: What was the strategy that you used? 

S: I just eliminated. " It couldn't have been 

how far it, was from Polaris because it never 

even mentioned that; and notify the U.S. Coast 

and Geodetic Survey? That's where her father 

worked, and didn't have anything -to do with it; 

t and send news to King Frederick — they didn't 

send news, he found out himself, 

I: itfow^ based on the discussion that' we've just had 

about this question, do you consider it to be 

S: It's not unfair. ^ 

I: Would* you predict that somebody Who didn't read as 

well as you did would have a problem with it? 
S; I suppose they would. But if they just use their 

common sense and read the whole thing... ^ 
I: By common s^nse, you mean going through each* of the 

possibilities and ticking them off? 
S: Yes. 

Quite often a request to verify an answer led a high achieving 
student, through the justification, of a process of elimination. By 
contrast, the same request of a low achieving student provoked a quite 
• different response. For instance, a 5th grade low achieving male (GJ) 
f correctly answered the following q*^stion from the Novelty passage 
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(CAT 18C) by selecting the 4th alternative. 

Why does 'the speaker ask if life has 'robbed you' or 
is 'getting you down'? % * 

a. to ridicule the listener's feelings 
bi to infonft the listener of dangers to avoid r 
c% to persuade the listener that no risk is involved t 
d. to suggest that the listener is missing out" on * -, 
something 

The expected answer is cued most importantly by the follotitag passage 
sentence. 

If it , (life) is (getting you down) , we can offer you 
a wa^ tio'^come alive by putting a surprise in.- your l±fe. 

The interview began with a request to verify the answer. s< * 

* I: Why are you thinking that it's 'd^ and not one of" 

the others? . , . 

S: Cause it's asking you if life is getting you down, 

to make you think of all the bad things of life, 

and try to persuade you to get this product, and that 

wilUkeepL you from getting robbed. 
I: *What* about ''b', since you said "the bad things in 

life?" ^ 
S: I thought that the salesperson *is trying to keep 

you from buying anything else. 
I: You mean lik^ buy another product instead? Well, 

what, about 'c' then. Maybe that makes 'c' a better 

answer? 

S: Oh, ye&h. 'c' looks good, 'c' is probably better. 
I: Now explain to me why you think 'c' "I3 better. « 
*S: Cause he's trying to persuade the persbn. ~ 
- X: O.K., but does he actually* say something about no 

risk involved? , What kind of risk do you think 

he's talking about? ' 
S: It says 'getting robbed' and a lot of people g« 

out and buy stuff and they get robbed. 
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I: Oh, I see. And he's saying that this wouldn't happen 

in this case, so there's no risk* 
S: Yeah. 

> . The change in position wh ictf the interviewer was able *to negotiate 
in man Z 9 t ^ ier c *s es indicates that students can be confused 

raX^i^f^tn helped by attempting to verify their answers. In t^ils 
case the student had not grasped the figurative reference of iAAf e 
robbing someone ^ and had used his own conception of it to rationalize 
another Answer. ~ 

While it was possible to redirect some low achieving students to 
jreco^sider their first choice^, , other students with the s§pie reliance 
to verify expressed their unwillingness in a refusal to cons ideifift her 
alternatives, although ^t was often c^ear that they suspected options 
oth^r" than tt(e ones they chose. might be correct. The following 
dialogue /with/ a third grade low achieving girl 
demonstrates jihis point. 




Does one of thenrseem like It might be better? 
How would. you decide? If somebody says, "I like * 



this one," aftd «somegody. else says "Oh, I like this 
J • ofle," would you haye a" fight about it or what? 

S: That person goes, with his, and I go with mine. 
I: But; j^Ly one "catr.be right, remember. \ 
% S3 ><J^Jrd fihd^ut which one is right and which one # 

• f is wrong. ■ n 

I: What If^you were wrong? Then what would you say? 
S: * I wouldn't s$y .anything. I'd^j-ust s ( ay" I was 
wrong. * 

^t is" certainly not assurance which motivates the objiurfccy (J^this 
.student's positions, it is resig^alti^n^toV belief- thit tests are £ 
somewhat^ capricious. Students* in this category feel incapable of resolving 
the conflict ilfeie^ent in a set of alfeyer options in a way which will' - * 
invariably lead them to the expected answer, and so they give their 
allegiance to the answer which' f±ts*be % st with the schema £hey have 
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elaborated for comprehending the story. To their credit, it rarely 
happens that they are unable to provide some plausible rationale fojr 
their choice., just as GJ did in the example cited earlier . Answers 
given rarely seem to be without some basis in fact or imagination. * 

SrhpjML Activatiotf ^ \ 

• , The*dif f iculty that some low achieving, readers have in verifying 
their selected answers marks the lack of a me£acomprehension skill, 
rather than a comprehension skill per S£. The low achieving readers 
we studied, 9 in fact, gave every -indication that they invoked the same 
generalized reading strategy — invoking, activating, and reformulating 
an underlying schema — as 4id the, high achievers. A difference did 
show itself , however, in regard to how raurch dependency was accorded 
the underlying schema. Low achieving students generally seemed less 
willing to foresake or drastically alter their schema in favof of 
investing in a new schema^, .while the~high achieving students typically 
showed no such reluctance. This pattern of results is consonant with 
our other findings particularly the difference in processing strategies 
discussed earlier. A reader who approaches new material one sentence' 
at a time is forced* to assess that information in relation to some 
underlying conception in order to reach conclusions which relate new 
Information to itself. In other words, the integration of information ~ 
which is presumed to take place during hietfarcKItal processing is ^ 
achieved during detail processing through the mediat^^i^f an under- 
lying schema. If the schema is accurate and full, tnW the new 
informatiBl presumably finds its properly integrated spot without the 
type of reading implied by hierarchical processing; in which each 
statement is evaluated first for structural relationship to other 
statements and latexL--for its fit to a preexisting schema. * 

Possibly 'either p toe ediire can produce full comprehension, although 
the methods for demonstrating comprehension would have to be sensitive 
to the procedure used. However, if the reader f, s existing sfchema is 
only weakly developed, comprehension will T>e more difficult for detail 
processors, *since their fit of the new data is more likely to be 
erroneous. Evidence of this claim can be found in many interviews 
and at all grades, but it is especially evident with the youngest subjects 
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For Instance, a 3rd grader had difficulty answering the, following 
question from the Ruth passage (CAT 13C) » 

Why was the copy of the map made out. of metal? m 

Sh6 begag the discussion ^of the item by remarking that the passage 



di^^t tell why — Which, in fact, ±t does not. It states 

Two years later, Ruth Asawa made a lasting metal - 
copy of the paste map. 

making the reason dependent on interpreting the word 'lasting 1 as 
inferentially related to the cause. During the interview, the student 
on being asked why she though Ruth used metal for her" map immediately 
and without consulting the passage selected the expected answer* 

* So that it would last a long time. 

The interview then continued from this point. f 

I: Explain to me why that (choice) would work? 
S:* Because " if there was a fire in the schooi, 
' the thing would just get hot, it wouldn't 

burn or anything like that. 
I: So it would last a long time. Could it be «r 

like to the thftd (choice)'- so that many 

» 

people could work on it? Hpw would that 
^- work out? 

S: If many people had a whole bunch (^equipment , if 
they drop stuff on- i^, it probably\^ould 'dent , 
but it wouldn't ruin the map. r . „ 

J: O.K., so you can figure out £ Reason for that one 
too. Which one do you think is^'the best one? 

S: I think the third one (i.e., people working-on it). 

% * . J • 

I: Remember, where they talked about the metal map in 

/ the story? Was it the beginning, middle, end? 

6: I thirjk^it was in the middle. * * 

(Student had a great deal of difficulty locating the reference using • 
her scanning procedure, but after some period of time finally found it.) 
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S: Oh| here it J*. .Last paragraph. It Says, 'Two-years 
later Ruth Asawa'made a lasting metal copy of the paste 
map^V * , 

What* do yoU think i£ going to be the right answer? 

It's g<4ing to be' the first one \i. e. last a long v t 

time) ' ' ; , 

Why? " • ' ' 1 - 

Because it- s^ays 'lasting 1 - lasting a long* time. 

This interview,, besides indicating the student's reliance on her ' 

own background information in answering the question, illustrates 

several concomitants of adopting this approach. First, she is unable , 

to locate information based on her understanding of the stbry eventfi, 

although she has a clear recollection of the mention of the metal map. 

^Lecond, in the absence of an overtly jstated reason for making the map 

.out of metal she contrives one reason, then 'another , and weighs 

the probability of eafeh being the preferred choice against criteria 

external to the story. In first selecting the correct answer, notice 

that her reason is peripheral to the 'evidence as presented in the 

passage, which, in effect, means sh^ has selected it, if not entirely 

by chance t , then fdr the wrong reason. As an indication of her t^rue 

passage coiJ^rehension, then, the question is uninformative.. / Third, 

'she imports information from her underlying schema into her conception' 

of the sto»ry # making reference to the possibility of, a fire, to some 

unspecified equipment, and' to the setting of the events in a schooll 

Finally, she rejects^all of the aforesaid logic in favor of an explicitly 

stated reason which intrudes itself during a verif icational procedure. 

♦ 

The procedures which this and ^ther students have followed in 
reading test passages do not necessarily comprise a poor comprehension 
strategy. To the extent that they involve the student in restructuring 
a schema they perform just the task they should as a reading event. 
However, to the extent they deflect the student from selecting the «~ 
expected answer on a test item, they fail to achieve their specific 
purpose. If these suggestions can be independently verified, then 
it would seem that reading comprehension tests are either biased 
toward the hierarchical processing strategy or are measuring non-* 



comprehension^ tasks. In either case the student who has successfully 
leayn.ed to cbpe with this form of instrument would be at an advantage 

'in. comparison to one who has not. 
Test-taking skills 

The analysis of the interview data provided several independent 
examples that, comprehension issues aside, the CAT was also measuring 
test- taking skills. Certain of these skills, of course, are necessarily 
involved in taking this or any form of structured' test, and to a certain 
extent our educational system' has subtlely or overtly prepared 
students to develop them from an early age. The comprehension tests * 
recognize the contribution, that test-taking ability can ma^e in * ? 
affecting a student's score, but they maintain that these skills arte * 
sufficiently taught and leariled by means of the instructions to the 
test and by the administration of practice problems accompanying 
test. Our discussion here, however, will exemplify the kinds of 
skills which are rarely taught, but which must be grasped if students 
are to succeed on tests. 

In a common type of problem a test item sets up the possibility 
of two logically correct options, one of which i^ literally verifiable, 
the other inferentially verifiable. In this situation there is an 
unwritten and untaught, rule that the literal answer is to be * 

.preferred over the inferential and should be 'selected. Students when 

confronted with this kind of choice either recognize the difficulty 

and weigh the alternatives or fail to recognize it and *go with one 

or the other based on their understanding of the passage. 
* 

A particularly troubling item was included on the Plants * 
passage (CAT 15&). It' rea<r . • 



Why* does the ^sundew move its. tiny red "arms' 

a. - to jatch insects 

b. to spread a sweet smell 

c. to feed n^ young plants \ 

d. to gain an insect's attention 

The passage literally supports only the first option through a link 
with the passage theme, but the fourth alternative is inferentially 
possible also. One 5th grade high achieving student (CM) immediately 
noticed the difficulty. 

^ J 



S: f A f and 'd'... It says f It has red vines that 

look like amis, and these vines are covered with sticky 

drops. 1 And it could be to catch the insect or it 

could be to gain its attention. . .Well, f a f \>r 'd,' 

but most likely when they're ^talking about trapping 

insects (as) the main thing of this (passage), f a f 

' would be it. But if it wasn^t_(the main idea), if it 

wife just like a story didn't say really any 

* » 
specific topic, then it could be/cf. ' 

I: Then you would answer f a f on the test? 

. S: They're both tricky. I don't know which one right 

now. f D f seems to be mdre \mat I'd pick, because I 

don't think the sundew waves its arms to catch 

insects. I think that's just its natural way of 

doing things.... But moving its arms isn't what 

cratches it, it's what attracts the insect to come near. 
* 

A second, low achieving 5th grader (MT) also saw the dilemma, 
but focused on 'b' as a possibility, because he assumed the sticky 
drops on the sundew's arms exuded a smell. When alerted to option 
'd r , he also admitted its possibility. Unlike the preceding student, 
however, he had no reasoned procedure to ,guide him out of the problem 
and admitted finally that he wo\*ld probably mark two answers correct. 

A second type of commonly encountered test-taking difficulty 
concerned CAT passages which presented a position advocated by some 

r 

person other than an unspecified narrator. / Certain items based on 
these' passages asked the reader's judgment regarding a particular 
claim 'made by the advocate, but in doing so did not specify whether 
the reader was to judge the claim from the standpoint of the advocate 
or the objective bystander. The Andrew passage (CAT 15C) provides 
a good example. In one item the reader is aske< 

+ Who said that "Creature^ in Ink" is the best display 
the school has ever had? 

a. Andrew Warrel 

b. everyone who saw the show' 

c. the editor of the Oliver News 

d. all the students at Oliver Schoo^ 



The passage provides the following information. 

The show is titled "Creatures In Ink."'. . .It f s the best 
show Oliver School has ever had. Everyone who sees 
will agree. 

One "high achieving 5th grader (MB) responded to the question as follows. 

S: Well^fcM say this is f a, f even though it could 
- - * be ~a lot of the other ones. . .Because you can 
tell from the children's enthusiasm that they 
think everyone who saw the show would think it's 
the best. And all of the students at Oliver School 
think it's the best. (But) Andrew Warrel, is the only 
one who really says it. 
I: How difficult would you say this (question) is? 
'S: This one is difficult in a new way*. I've seen « 
% a few but not very many that are difficult in this 
way, where you could have a lot of possible answers. 

A low achieving 5th grader (MY) initially chose the expected answer 
but could be dissuaded from it to the other option. 

S: f A f , Andrew Warrel. 

I: Are you sure its not 'd' (i.e. all the students 

at Oliver School)? 
S: Oh , yeah, v d f because all the students are saying 

it f s good. 

I: Originally you said it was Andrew Warrel. Why did ' 

you think it was Andrew Warrel to begin with? 
S: Because Andrew Warrel is writing to the editor. 

5 

Both of these stu4ents perceived the .difficulty in answering the • % 
question without a declared point of view, but whereas the high 
"Achieving student resolved the difficulty in favor of the objective* 
bystander position the low achieving student resolved it in favor of 
the advocate's claim. We might hav$ predicted this pattern of behavior 
given the , predisposition of the* low achievers to stick as closely as 
possible to the stated message. Still, it is cj\aite evident that the 
probability of the expected answer had not escaped the low achiever, 
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meaning that her 'wrong 1 answer cannot be construed as a failure 
to comprehend the passage, only as a failure to ascertain the test 
maker's Intention. 

Hypotheslged problem type , * . 

The opening remarks to this chapter mentioned that the pattern of 
responses to the ten problem types identified in the preceding chapter 
was uneven, that some of the types did not produce the hypothesized 
difficulty for any identified group and others were not consistently 
difficult for any single individual. *» Furthermore, of the types which , 
did prove difficult the responses showed that the burden of the difficulty 
was borne by the low achieving students. The high achieving readers, 
while often noting the potential for confusion and error, nevertheless^ 
claimed they had the skills necessary, to avoid answering incorrectly. 

* The status of each hypothesized category as an actual problem type 
is provided in Figure 3^-1. 

Figure 3-1 

Hypothesized problem types and determined status as an actual problem 

Unsubs tant iated Ver if led 
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1. plausible distractqr x 

2. difficult inference x 

3. unstated point of view % x 

4. hidden detail x 

5. reliance on background information x 

6. inadequate identification of referents x 

7. vaguely worded questions x 

8. absence of superordinate topic x. 

9. passage independent questions x 

10* no best answer x * 

11. unknown vocabulary in question - x 

The status is listed as either verified or unsubstantiated. A verified 
problem, is one which consistently proved to be difficult for students 
or which provoked substantial discussion. An unsubstantiated problem is 
one which, contrary^ expectation, proved to be easily answerable or 
to arouse no recognition of difficulty. ' In add^ion to the ten 
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hypothesized problem types discussed in the preceding chapter, Figure 
3-1 also includes reference to one additional problem type which was 
unanticipated but recognized and verified during (he interviews. In 
the following discussioA the unsubstantiated problem types will be . 
discussed first, followed by consideration of the verified* problems. 
Unsubstantiated problem types 

Hidden detail. None of the CAT questions identified as involving 
a hidden detail, including the question from the Sequoya passage 
exemplifying this type (p. 31 ) , proved at all troublesome to any of 
the students interviewed. Moreover, attempts to point out the" hypo- 
thesized difficulty of such questions were largely unsuccessful. Students 
seemed to elevat^the status of the queried information to accord it 
the actual importance of Its position within the passage. For instance, 
the Sequoya passage question places what we imputed to be -an incidental 
detail — that Sequoya' s written language contained 86 letters — in the 
elevated position of a passage assessment. One high achieving 5th grader's 
(MB) response to this out -of-cont|xt mention was to give more weight to 
the information. In response to a question asking why he thought the 
information important at all, he said 

S: It's always important to know the amount, and 86 
signs tells you that he must have worked hard 
enough to really do a thorough job. of the alphabet. 

What he had done was t;o assign an importance to the information con- 
sistent wit'h the discourse category in which it was included. 

It is not as easy to ascribe the same logic to low achieving 
readers, who generally were not vocal about answers they were assured 
of. However, given their inclination to detail processing it is 
possible that the fact of 86 letters would be precisely the sort of 
information which would capture their attention on its own right. In 
fact, several of these students answered this question without any look- 
back. - ; 

Reliance on specialized background information. This category 
proved to be insufficiently defined to distinguish it from other * 
verifiably difficult or unsubstantiated problem types: In certain 
instances the background information required of the reader was 
sufficiently generalized that all of the students interviewed knew the 
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information — for ' instance, the information that a grapefruit is a fresh 
fruit. The problem of an imprecise definition revealed itself especially 
in cpnfusioris with unstated point of view category,as lor instance, in the 
question from the Andrew passage discussed akove (p. 30) 

Which of tlie following is probably the main reason Andrew 

Warrel likes the~art show so much/ 

< 

Originally this \*as categorized as requiring the specialized background 
information that -writers may-not be entirely objective in advocating^ 
a particular position or opinion. The correct answer, ^rtiich asserts 
the reason to be Andrew's membership in the club sponsoring the art 
show, however, was later felt to b e e better characterized as a failure 
on the^part of the test developer to clearly specify whose point of view 
the student was to take in answering the question. 

, i x A Second merging of criteria was found between this category and 
the category of hidden detail precisely because the information necessary 
to answer the question was so diffusely stated that the reader was as 
likely to reach a decision tfn the basis of background knowledge as on the 
basis of Information presented jLn the passage. For instance, a question 
in the Ellen passage (CAT 13D) a?ks the reader for the meaning of the 
word 'nearsighted* the information for which is not explicitly given -in 
the passage at the point where it would most likely be expected, prompting 
a conclusion that the question required background knowledge. 

, Vaguely worded questions. This category, like the preceding, is , 

'also inconsistently defined. In certain cases, such as the question 
from the Ruth passage discussed above- (p. 33 ) 

Which word best tells about the boy? 

most students were able to retrieve the correct answer. In other 
cases, however,, a vaguely worded question did prove troublesome, but 
this may have been a consequence of the intrinsic difficulty of the 
information content of the question rather than a factor of its 
structural form. In general, it seems that when faced with a question 
categorized as vague, the student is fofrced into a process of 
elimination strategy during which one of the options tends to stafrd - 
out more prominently than the others. The student 'will select this , 
option, but will oftentimes express some doubt or uncertainty about 
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the choice. These students will also sometimes express their dissatis- 
faction with the fairness of the question. ^The following interview 
with a l<>w achieving 3rd grader (EB) on a vagtrely worded quest ipn 
from the* Cosby passage (CAT 13D) illustrates these points. The question 
read: - 

Bill Cosby f s stories show that he probably was 

a. happy about being youtig v . 

b. excited about watching T¥^ I 
worried about his brothers. 

d. curious about making records. 

The student initially found it hard to pinpoint an answer and the 
interview followed him ^irough his testing of the various options. At 
a point when the dif f idbTFy of the question was entirely evident to both 
the student and the interviewer, the interviewer asked 

, I. What makes this dne so hard to answer? 

S: I don't know, i v 

I: It's a hard question for you, then, right? 

S: Yeah. \ 

I: Is it a fair question? 

S: Yeah. 

I: Why is it fair, if its so hard? 

S: If you read the story, yoa should know the answer. 

I: You did read the story, and you've been telling us answers 
all along. How come this one is so hard, so much harder? * 

S: By the time of the last question, you always forget 

* o 

the story. 

I: Maybe that's a good reason. What about if you just 

guessed? Which one would you guess/ 
S: Probably 'a' (happy about being young).' 
I: Why 'a'? ^ ^ 

S: 'His stories helped everyone remember what it was like 

to be young. ' 

I: And that comes pretty close? V 
S: Yeah. / A 



I: Is it a guess or is it for sure? 
S: It's a guess., 

I: Do you think it f s fair for them to ask you to make 

guesses like that? 
S: No, because you might get it wrong, right, wrong, right.. 

Passage independent questions. Students interviewed only rarely 
regarded test questions as being passage independent, a£ least until 
the possibility was specifically addressed during the interview* Even 
in 'this case, however, students almost overwhelmingly felt that the 

selected answer should be verified, either tHroiigH"To<Jkb^:ck" to -the - — - 

passage in the case of the better readers or through some^tdgically e 
defensible connection to passage information in the case/*of the poorer 
readers. In any event, given the experimental procedure, all students 
1 did in fact read the passage before they were given the question to 
answer* ^Xbis would preclude the possibility of reading and answering 
a ^estion totally independently of the passage. The points made are 
illustrated in an interview with a high achieving 5th grader (KE) on 
the question from the Andrew passage exemplifying this category fr. 34 )• 

I: What about if you didn't read the story at all. Could 
you still answer it? 

S: Probably. But it's better to read the article so you 
know what you're talking about. But you could do it 
without reading it. 

I: Did you answer without reading it (that iS, without re- 
reading it)* 

S: Yes. 

I: Is it still a reading comprehension question, if you 

don't have to read the passage to answer? 
S:- Well, it's not really reading comprehension, but it could 

be put there because it is about reading, and there's 

not quite another place to put it. 

9 

, No best answer. Students tended invariably to approach questions 
which had been categorized as not including the best answer among the 
options as if they were simply more difficult questions in which the 
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best answer among the options was elusive. At most, students could be 
led to seeing the questions as unfairly masking for distinctions which 
were too finely drawn, a consequence of none of the options being, 
directly ot» target with the information in the passage. In situations 
such as this, the low achieving students especially tended to fabricate 
the justification for selecting the answer they chose. More oftfen than 
not, however these students were able through their comprehensive under- 



standing of the passage to select therirtem which most closely approximated 
what the best answer should hav/teen. Consequently, few students tended 
to answer these questions incorrectly for the hypothesized reason. 

Verified problem types 

Plausible distTactor. The category of questions for which one or 
more of the distractors could be judged correct, and plausibly argued 
in the final analysis constituted the most-frequent type of problem 
students had to contend with. In many of the examples discussed earlier/ 
in this chapter in connection with processing strategies and schema 
activation, low achieving readers generally and high achieving readers 
in some measure consistently demonstrated difficulty with questions 
labeled in this category. In many cases these* questions were initially 
answered incorrectly and in others students could be made to see an 
alternative position when it was drawn to their attention. On the 
question from the Ruth passage cited earlier in this chapter a student 
was shown to have chosen the correct answer for the wrong reason. Notice 
that in the follow^Ktg interview with another low achieving 3ti grader 

4 

(AC) essentially the same type of analysis leads instead to the selection 
of an incorrect answer. To recapitulate, the student was to decide 
on the question of why the map was made out of metal. The student 
responded 

. • S: 'So that many people could work on. it. 1 
I: You mean 'on top of it 1 or what? 

S; Yeah, like if you have a big map and you write on it 
where you J re going to go and how you're going to get 
there. 

I: I seek. Alright. 
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I: Does it have to be made out of metal for people to work 

on it?. * 
S: Not necessarily. 

= K 

I: What is the question asking you? 
Sf It's asking why was the- map made out of metal. 

It f s metal because if you're driving along and you 
don*t have any mats or anything, yoju don't have to have 
a table you can just write on it. 
I: Where does it say that they were going to put the metal 

map? Does it say. 
S: I forget. 

I: See if you can find "it. Where do you think it would be. 
S: It didn't say anything about where ^they were going to put 

it. (Finally ^locating the reference). 

Oh, outdoors, they were going to put it outdoors. 
I: Now, if they put it outdoors, do you still think that 

answer is the right one, sq that many people could wotk 

on it? See what I mean. 
S: Yeah, I see wh&t you mean. People didn't have to work 

on it, they could just see it, like, "Look at that", we can 
— ^ just see it; we don't have to work on it. 

I: But they still could if they had to. 
S: But tfcey could if they had to. 

I: Any other answers there that make sense to you at all, 

that could be possible answers? 
S: It could last long (so&nding skeptical). 

In one or two cases students even resisted the expected answer on 
quite "logical grounds. For instance, on the Balloon passage question 
cited in^Chapter 2 exemplifying this category (p. 28 ), a low 
achieving 5th grade student (15b) rejected a suggestion that the first 
balloons were piloted for sport, initially saying 

S: There's not too much on that part. 

The interview continued 

I: When they talteed about J.A.C. Charles and Letitia Sage 

going up in balloons, why,do>you think they went up? 

S: Just to see if it would work, since it was new. 
> 
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5: 'C'7 Uhat^is, instrumei 
I: Or do/sn'^rfquite.fit? 



I: Is that ,the same* as any of these over here. 
| -S: f C'? # (that^is, instrument testing)? (Sounding doubtful.) 

I: 

Not really. 

The lo&ic of her answer' is quite defensibl%*4ijom the cohteit provided, 

but it seems to preclude the .possibility of choosiqg the expected answer. 

Difficult inferences. This problem type was identified much less 

frequently than the preceding type and, overall, constituted less fef - 

a difficulty. We have included it as a* .verified problem type, however, 

, - • • *• • m( + - * * m 

because even though students were often £ble to f irid t Heir "way* to the 4 

cotrect choice, this generally happeiJfetf'b^ default — a consequence of 

the question having clearly inappropffHfte distr£ctor#. ~ It was mainly / 

those students who did not use a process of ' elimination strategy in - * 

selecting their choice who answered incorrectly. When v a particular 

question, under this category lent itself to the possibility of sel^ting 

and jus£Lfytog one of the distractors, as did r for instance, the , 

question from the Maria > passage exemplifying this type (p. 29 )% this 

category becomes "effectively indistinguishable f Jrth the category of 

plausible 'distractors. The main difference is that the particular cause 

of the student's confusion can be more precisely attributed to a structural 

inadequacy In the passage. This isNq^de clear in an interview with a 

low achieving Jth grader (CK) on. this question. Th^student initially . 

.answered the question incorrectly. * ' • ' 

S: # |B' probably, 'complete thefr calculations of the 

cqjiet.Jy r ' 

_i • " T: Do they a^ually tell you in the story? * 

£; No, Just that They sent him stuff, I think. ...It doesn't 

. ^ . say what she, sent it for. 

I: How do you think they expect you to find out? Do they 0 

% giye you any clues? # ' 

*' S: She just sent it and he sent-back goQd news. 

^ % " I: Is that a clue? \ ' ' , * - 



. S: Yeah,, orobably. be^ij' (that is, the -correct answer). 



ERIC 



The*studfent was able to be guided to the correct answer indicating ^ 
that the necessary information was there in the passage," but' that it 
was somewhat 'inaccessible because of placement and structuring. In 
otfoer words the information was apparently not highlighted enough for 
the student to* see the possibility of ,arf inferential link on the finst 
pass. , ' 

" Unstated point of view. Examples of student difficulties with 
questions ujider thiB type were presented earlier in connection with 
the discussion of" test-wiseness. 9 • *~ « 

Inadequate identification of referents. This problem type was 
verified as a true difficulty mainly for younger students. Even by 

• the 5th grade students appeared able' to skirt the * inconsistencies entailed 
in treating an inadequately specified referei^ as, clefinite^ Third 
graders, however, were oftefl misled by the lack of specification into 
^^uming ttle question pertained to a passage character other than the 

.one intended. For instance, 6n the Ruth passage question illustrating 
th,is type (p. 32 ) two 3rd grade low achieving students (DF, AC) had 
the same reaction to the expectation implicit in the question that the 
character under consideration was specifically identified in the passage. 
Both assumed "the girl"* Was Ruth" Asawa, when in fact she was ah un- 
specified friend of Ruth's. DF initially chose an incorrect answer, 
but on lookback she changed her an$we*r to the one expected. Even so, 
howevet, her understanding was insecure. 9 - ' 

I: Do you remfember the girl they're talking about? Who f 

is %it? Do they tell you? " * . 

S: I don't know.- Oh yeah, the artist Ruth, .she took a 

# 

wooden knife. t - 
Ij The girl they're talking about is Ruth then? 
' - S: There's also Another girl and a boy. 

I: Is thi^girl the same as Ruth? " * 

S: I don t know, * . 4 

1 

I: What do you think about ^the girl they're . talking about 
here. Is that- Ruth? * 
• S: Yeah. ^ 4 

Absence of .superordin^te topic. Questions in this category 
occurred relatively infrequently,* although they reliably- led to 
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difficulties in interpret tog thw force of a question. For instance, 
in the CQsby passage exemplifying this type (p. 34* the absence 
of an explicit topic statement on which to hatog tjie information led 
students to interpret the question as asking for a statement of fact 
rather than a statement of purpose. The following interview with a low 
* achieving 3rd grader <EB)*illustrates this point. 



S: I don't get' that quesTftbn, 'The story says £hat Bill 

Cosby has been on TV shows to.... 1 
I: What do you think it's asking? 

S: 1 don't know.'^ Something like What's he do on TV shows. 
I: If that's what you think it means which one do you think 

is the right answer? Which are you thinking about? 
S: I'm thinl&ng about all of them - play football, I'm no't 
* * sure of; making records. . .well, * yeah, he sings some of 
his songs ~on TV shows. 0 
I : So you think that might be 1 the answer? 
S: Yeah. ' • , 

I: If the question here means something a little bit 

different - if it means 'Why does Bi^l Cosby go on TV' 
S: I gues3 because the studio likes him.- 

The strength qf thjB student's initial interpretation of the question 
evidently makes it difficult for him to pick out the correct answer even 
when specifically cued. " 

Unknown vocabulary .in question . v This category was initially not 



' no 



anticipated, but, even^ if ±t had b^en, there would have beetr 
to reliably predict which questions contained critical vocabulary. 



unknown to the 'young readers. Evidently,* however', the controls the 
test developer^^e to assure that vocabulary whic*h cannot be inferred, • 
from the passage cdntext is eliminated sometimes 'breaks down. 
% • When the problem revealed itself among the answer options students, 
particularly those among the high achievers, were not inclined to choose 
items whose meaning they, did not know- Presumably ,• they /assumed that, 
if a vocabulary item was unfamiliar it was one qi the distractors. They 
then set about trying to verify .another option as the correct answer « w 
An interview with a /high achi'evinjj^ltji grade^ (AEJ on a question from 
• .the Plants passage illustrates these paints. The ptudent does not know - 
the meaning of the expected answer, • the ward 'haj 
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j*S: 'From the passage one can tell that most insect-eating 

plants are very.,. 1 What does 'hard^' mean? 

I: Well, before I answer you, v go through the other 

possibilities and see whether any- make sense. 
* ■-- . ♦ 

S: O.K., well, one answer could be 'b' for the bugst for 

the insects, pause the reason they went to the plants 

is because* they smelled something sweet, and it says 

'tasty', and then they just catch the insect. 

I: Now, 'h^rdy' is a word that means strong, and able to 
withstand hard conditions. 
/ # S: That' would be another possibility. ° ' 

It Is one better than the other? 

S: Well, I guess I'd go with 'a' (hardy), 'b-tasty,' that's 
what sort of lured the ing^^s to "the plant, because they 
0 smelled something swe^ 

I: Which one are you god 

* S: Probably equally good^^Fcan't make a decision. 
» I: Now,JLf you didn't know what 'hardy' meant. 
S: Then I'd go with 'b'< 

On analysis the interviews reveal that students are not capricious 
in choosing the answers they do to test questions. Furthermore, evidence 
of serious comprehension difficulties is lacking. We have seen reason 
to believe instead that low Achieving students evidence a lack of 
metacompi;ehension and test-taking skills sufficient to successfully 
find their f way through the structural and lexical difficulties which 
the tests themselves preseilt. Under this interpretation good readers . 
are those whb have an ability to process ptoorly written prose. It 
remains , to be seen in the next chapter wheth^nwe can substantiate 
this claim bj demonstrating enhanced performance when passage material 
is rewritten to*improve its structural cohesiveness. 




Chapter 4 
.EXPERIMENTAL RESULTS 



The interviews indicated that low proficiency students apparently 
can access the same comprehension abilities as high achieving students, 
but that they lack or do not exercise verif icational routines to assess 
their choice in a fuller context. Because materials are sometimes 
written in vague, ambiguous, or misleading ways, the verif icational 
procedures become as or more **impo*t ant than comprehension routines in 
selecting the wanted answer. This suggests that the tests are measuring 
some test-taking skills iiT addition to conrprehe^ion skills. It also 
suggests that if the various ambiguities, inconsistencies, and in- 
accuracies are reduced or eliminated the balance would awing more 
toward the assessment of comprehension skills than tfhe assessmeht of 
test-taking skills. To this poiht the experiment described in Chapter 
3 was designed. The expectation was thatf 4>y rewriting passages from 
standardized reading comprehension tests we could show enhanced^per- ^ 
formance for low achieving studeitfr^. 

Test Objectives 

" * * 

Several considerations had to be met in designing- the experimental 

test to assure* that it did not radically change the character of the 

original test. It was* necessary* first to assure that the objective of 

h test question was held constant} ;^or example, that a question 

calling for ian inference remained an inferential' question for the re- * 

written passage as well. It was, of course, difficult to control for 

subtle differences in the obviousness of the inference, since in many 

cases the reason for an ambiguity could be traced to the inadequacy 

of the discourse in providing sufficient evidence of a discourse link. 

For example, in the Sequoya passage (CAT15C) a question asks *rtiy it was * 

harder for the Cheroke^s to talk tpgether after they had moved to" the 

West. Part of the information cueing the correct answer — that they 

lived farther apajrt — must be inf erentially linked up. ^ In the 

.relevant part of the original version, the sentences 
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The Cherokees were pushed from the South to the West. 
Their homes were scattered. ^ 

leave the temporal link between the moving and the scattering inexplicit, 
leading to possible interpretation that the two events are simultaneous 
rather than sequential'. In other words the scattering could equally 
apply to the homes in the South or to the new homes in the West. * In 
the rewritten version, the sequential connection is made explicit. 

The Cherokees were pusfred from their homes in the 
4 South and forced ta scatter to the West. 

The change removes the need to make this inference, although the in- 
ference to the ^proposition of the Cherokees having difficulty talking 
to one another remains intact as does /the lexical inference 'between 
living^farther apart and scattering. Consequently, the larger compre- 
hension objective, though modified, is essentially the same. ; v 

The desire to keep the question objective intact, however, requires 
an assurance that the objectives across a^l passages within the same test 
are Consistently applied an^, furthermore, th£t all of the tests define 
their objectives in a similar manner. An exanflnation of the tests showed 
that neither of there assutajices is warranted. On the first point it 
sometimes happens that a question declared to test a certain objective 
actually meets the criteria associated with another objective. For 

-instance, in the MAT (Elementary JS) it is claimed that the question 
* 

- Maple Syrup is made from tree: 

a. bark 

b. branches / M 

c. sap . 

d. leaves 

tes£s literal comprehension of specific passage details, based on the 
reader's use of ,"inf ormation^Stated explicitly in the passage" (Prescott 
et al. 1978:55). However, the information. necessary to give the expected 
answer — sap~*-is presumably, contained in the sentence 
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Maple sugar comes from the sap that runs through Maple trees. 

which makes no mention of maple syrup. Tj^ reference to syrup is contained 
in a separate sentence following a discussion of how sap' is bQiled to 
produce it. The reader must make an inference drawing together information 
in different mparts of the passage in order to correctly answer the question* 
The true objective of thip question, then, is closer to what the MAT 
4efines as inferential comprehension of global passage information, namely 
the use of information "implied in all or a large section of the passage 
to infer answers to questions telated to. ..cause and effect 11 (Prescott 
et al. 1978:56). '* . , 

On the *second point the Cferee tests examined cannot be said to utilize 
the same objectives, and although superficially there seems to be at 



rficially the 
:tives, l±t is 



least some agreement on the broad objectives, lit is apparent that the 
definitions of these differ. Figure 4-1 -lays out the 'main objectives 
of the MAT, CAT, and ITBS, givit^f or each an encapsulated definition and 
including "under some of them one or more examples of narrower objectives. 
The first point to consider is the general agreement that there are three 
modes of compreHension based royghly on the nature and scope of material 
the reader must process in order to answer a question. Only the MAT, 
which lists six primary objectives*, falls outside this pattern, but four 
of the six can be unified on the two parameters of*literal comprehension 
and inferential comprehension. Doing this reduces the total to four 
primary objectives, three of which align dlosely with the objectives on 
the other tests. We will consider the .fourth, vocabulary comprehension, 
at a later point. The terminology varies somewhat between tests — for 
instance, the MAT s inferential comprehension corresponds to the CAT s 
interpretive comprehension and the MAT's evaluation correspond^ to the 
CAT's critical comprehension and the ITBS's generalizations-but we make 
the equations because each test sets up the same number of distinctions 
and, as,£he category breakdown shows, each includes some common sub- 
object ive^.^ 

• * 

Closer analysis, however, shows that the main objectives are neither 
explicitly nor implicitly defined in the same way. The* explicit 



Figure 4-1 
Test Objectives 



MAT 
1. 

2.' 

3. 



vocabulary comprehension 

literal comprehension of 
specific passage details 

literal comprehension of global 
passage details 

main idea 

cause and effect 

inferential comprehension of 
specific passage details 

inferential comprehension of 
global passage details \ 
main idea 

evaluation 

real and imaginary 
author attitude 
propaganda 



CAT 

1. literal comprehension 

recall of facts 

2. interpretive comprehension 

main idea and Cause and effect 
solutions to problems 
inferred meaning 
figurative language (i.e. 

literary allusions) 
etc. 

3. Critical comprehension 

real and imaginary 
persuasive techniques 
author attitude 
* etc. 



word meaning from context 
explicitly stated information 

information based on all or 
large part of passage 



implied information ► 

information based on all or 
large part of passage 

Reader judgments relating 
, to mood and criticism 



actually stated information 



implied information 



reader judgments 
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ITBS 

1. Facts (literal meaning) 

vocabulary in context 

2. Inferences (interpretive meaning) 

caifse and effect 
characterization 

3. Generalizations (evaluative 

meaning) 

main idea 
author viewpoint 
figurative meaning 
. etc. 
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stated factual details and 
relationships (WH question) 

Why quest iony 



higher order skills 



definitions are first of all weak. They do /not inform the critie about 
the cognitive scope of an objective. We"* don't know, for instance, how 
to interpret the MAT f s definition of evaluaticfi— as a reader judgment 

0 

relating to mood and criticism — in terms of whaf the reader must actually 
do to solve the problem. Specifically what does he or she have to do 
in rendering an evaluation that is different from what happens when a 
literal or inferential comprehension question is answered? The examples 
which are included under each' category objective, such as an ability to 
distinguish between real and imaginary events or to perceive the author f s 
purpose, provide implicit definitions of the objective by constraining 
its range of applications. But even though the' examples provide more 
information as to the meaning of the objective, a comparison of th£ 
three tests at this level shows -a lack of agreement on where one or 
another of these subobj ectives should -b£ placed. Consider, for instance, 
the reader 1 s ability to grasp a main idea as a subob jective. In the , 
CAT this ability is considered an instance of interpretive comprehension, 
in the ITBS a generalization evaluation, and in the MAT an instance 
of either literal or inferential comprehension. There is certainly no 
uniformity of opinion on what comprehension skills underlie this 
objective. Other examples include the assessment of figurative language, 
which is considered an interpretive skill by the CAT and an evaluative £w~^ 
skill by the ITBS and recognition of vocabulary from context^ .a literal 
skill by the ITBS, an independent skill by the MAT, and a skill which • 
goes untested in the CAT. ^ 

)viously*the agreement on the components of the main comprehension 
skills is in some doubt, a consequence perhaps of our poor understanding 
of <£he entire comprehension process. If this is the' case, the individual 
tests^JJvbjuld not be faulted on their respective interpretation of what 
makes up a superordinate comprehension skill, though each of them can be 
legitimately criticized fo/r not applying their own criteria consistently. 



For the classroom teacher 
decisions, the best that c 
weakly correlate with real 



attempting to apply test results to* instructional* 

J 

an be said is that the test objectiyes may only 



abilities or real deficiencies. 1 
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For the experimental procedure the lack of agreement on test 
objectives and *the lack of consistency with which the criteria were 
applied meant that the point of each question had tor be reassessed from 
a more objectively derived standpoint. To do'thi^ we adopted a set of 
structural criteria based on the extent of the information which had to 
be appraised before ^he correct answer could be ippreheided. At the 
" same time we wanted to stay within the realm of tneca/egories as they 
were loosely laid out in the tests. 

The resulting analysis set up and defined four comprehension 
objectives. The first of thebe, literal comprehension , involves the 
assessment of information in \ single sentence of text J for example an 
MAT (Primary 2 JS) item asks the reader to complete the sentence 
Fresh mud was used to 

a. bake the bricks 

b. hold the bricks together 

c. make boxes for bricks 
^ d. put on the floor 

The answer is to be found in the following sentence 

The bricks were then taken from the boxes and stuck 

together with fresh mud. t / , 

For practical purposes, a sentence which contained all the in- 
formation pertinent to answering a question except that it used a 
pronominal marker in place of an identified referent was still considered 
t to be literally comprehended. Even though an inference has to be made 
to identify the referent, it was felt that this type* of inference is of 
low order and hence not critically affecting the designation. One bf 
many instances of this type occurs in the following example from the 
CAT (13D) . TKe question asks 

According to the story where did Bill Cosby play football? 
based on the passage sentence „ • « 1 

He tells stories about playing football in the street 
and racing; go-carts. 
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The Jie In the passage refers to Bill Cosby. , 

1 r 

Inferential comprehension involves the recognition and connection 

of information contained in two or more sentences of the. texts; for- 

t 

-example in an item from the Lawn passage (ITBS 7) a quest'ipn asks 

What makes some people think the lawn is soaked when 
pudding occurs? 

The passage Information states: 

Water either forms in the pedd l es- on the surface or 

just runs off. This is called "puddling. 11 When puddling 

pccurs many people think the lawn is soaked. 

Instances of this type are quite common and involve a host of purposes, 
although all of these are possibly unifiah*Le around the notion that they 
lead to an appreciation of the logical relations connecting individual 
sentences to the organization of the discourse. 

The third type, evaluative comprehension , requires the reader to 
appraise passage information in connection with background information, 
that is, information not available explicitly or inf erentially in the 
passage itself . truest ions in this category can range from requests for 
interpretations, such as judgments of affect, to information which is 
somewhat tangential to the text. An example of the firs\ type from the 
Sequoya passage (CAT 15C) asks 

' Which of. the following best describes Sequoya. 
Relevant passage information includes the following scattered sentences. 

> Sequoya,' a talented Cherokee teacher \was also 

' f deeply concerned. .. .Sequoya thought that if he could 

make a Cherokee alphabet, his people would bf able 

to write to* each otjier. He worked hard for twelvf 

years. . /.The Council -decided that Sequoya had indeed 

/ learned the secret of the "talking leaves." Within 
• > • • • 

v a few month^ Sequoya was able to teach thousands of 
. his peopie ^o re&<!C and^write their own language. 

The oorrect answer — inteiligent^-is ntf^tfiere explicitly mentioned or 
alluded to, so that, the reader must make -the judgment, on the basip of* 
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his or her own background knowledge,, that Sequoya's intentions and behaviors 
were in the class of intelligent .acts. * • 

The second type of evaluative question is generally , regarded as a 
literal type by the test makers who use it. It does, in fact, ask for 
factual information, but the information must be brought up frem back- 
ground experience. An 'example from the Australia passage (ITBS 8) asks 

The . bodj covering of the greater glider is most like 
-which of these? 



a. A cat 

b. A bird 

c. A fish* 



d. A snake t 

The passage provides only the following information. - 

Zoologists describe it as a "haity mammal , 11 which means 
it has fur ... 

» 

The question, consequently, requires the student to reflect on 

his or her background knowledge of animals and, by the convention adopted 

) ' 

here,, -becomes evaluative. r t 

The fourth and last category objective, main idea comprehension/ 
•departs from the others in not applying a strictly structural criterion. 
The notion of main idea, as discussed in Chapter 2, is primarily a 
pedagogical creation, rather than a discourse reality. ' In a strict 
sense aily idea can be the main one, depending on what interests and 
background the reader brings into the reading situation. A test-taker 
who has not been taught that a main idea is to be found in the various 
paragraph topics .will be at pome important disadvantage. For this 
reason the main idea questions in the tests were put into a separate 
analytic category, even though they juight otheS^se have best been 
'assimilated into £he evaluative comprehension catfegofy* 

The .resulting scheme of the reconceptualized test objectives is 
sunmarized in Figtire 4.2** Ibis scheme was used to reassess' the objective 
of each question used in the experimental test/ and forms part of the 
analytic framework for discussing the* experimental results. In numerous * 
case^ tl)£<j^suite'c( ia'the question being recategorized, most often in 
the jfi^ection of supposedly literal comprehension questions being reanalyzed 



into other categories. A full documentation of 
question object lyes is included in Figure 4.5. 



changes made in test 



Figure 4-2: Reformulated Test Question Objectives 



literal comprehension 



inferential 



rehension 



substance of question is re- 
trievable from a single test 
sentence 

question is answerable by drawing 
together text information inutvo 

more sentences 
question requires coordination of 
text and background information 
question focuses on one or more 
paragraph topics. 



evaluative comprehension 
main idea comprehension 

Test Questions > 

Besides leaving question objectives intact, a second consideration 

in rewriting passages was to leave the test questions themselves in their 

• *" t * 

'original form, even when these were judged to be poorly written or con- 

i 

fusing.. In such, cases, special consideration was given to clarifying, 
the question 1 a Intent by rewriting relevant parts of the reading passage, 
if .this was possible without introducing new information or specifically 
cueing the expected answer. In^^te instances, however, no intercession 
could meliorate the situation. For example, questions ufiose expected 
answer is an importation from background Information cannot be improved. 
The following question from the Sailors passage (ITBS8) is an example. 

Which food Gould best prevent scurvy? ^ ' * 

, a. beef 

b . grapefruit fc V 

c. ddughnuts ^ 

d. candy m . 4 "~~^-^ t 

> ' # - ' 

The expected ajyrwer — grapefruit — Is not mentioned in the passage and 

•* * ■» 

is cued only by the sentences 



The limes, like most fresli fruit and vegetables, contained 
Vitamin £. This vitamin both cures and prevents scurvy. 

Consequently, without actually making referdffce to grapefruits in the 
rewrite and violating the stricture on importing new Information into 
the passage, the answerability of the question could not be improved. 

A second type of impervious question was one in which the 
expected answer contains a critical vocabulary item which is unknown 
to the student and which is not easily inferrable from the passage. The 
following question from the Plants passage (CAT 15D) illustrates this type. 

From the passage one can tell that most' insect-eating 
plants are very 

• a. hardy • . 

b. tasty 

c. useless 

4* colorful 4 „ « 

This question, which was also used in the interview sessions, proved 
difficult for several students becau^ the exprQcted answer — hardy — yas 
not in their active vocabularies* and the {fessage cues were insufficient 
for them- to infer that this was the expected answer. Instead, these 
students tended to select the answer option — tasty— justifying their 

choice with the fallowing passage sentence. * . 

, ■» * , . 

The Venus f s fly trap has special juices on its le&ves. 
Insects la&d on its leavers to taste *the juices. 

The-assumption they mak^, and there seems no way to preclude its logical 
possibility, is that the pla**s are. .tasty to ittsects. Again/ there was 
no. way to rewrite the passage without explicitly cueing t^e expected 
answer, a tactic which would have changed the question objective. 

* Changes in the passage which w$re addressed to particular questions 
wefe made in the interests of clarifying critical points of information. " 
The objective was nbt to modify the question in its intent but to make 
it structurally consistent with the information presentation in the 
passage- For example, a test question from the Ruth passage (CAT 13C) ,asks- 

What did the girl us^to get paste out of the bowl? 



. • ^ . ..' ■ . r 

. expecting tflte student to utilize* the passage Information 

With a wooden knife, a girl took some caste from the bowl 
4 / and began to make "a cdfcy of tier schoo* ^ 

■< • ' ^ . ' ' -C 

The ^dif f iculty with J^he question lies in.iis presumption — indicated by 
the defidite article the — that some specifically, identified *irL is 
being referred to. A f ew ,st(idents Juring the interviews took this to 
mean that the question referred to Ruth in some way, since she was . the " 
only specifically identified "girl". In the rewrite an. attempt "was 
made to smooth^ the ijifconsistency by makii^g the person of the .girl 'in 
'.question more definite, without incorporating new information. This 
was done, by jilt roducing a sent6nce with topic force to prepare the 
reader for^the introduction of a referent and then tying the re^jent,. 
to the^ topic by a type^pf^ definite marker. These considerations resulted 
in. the^Cgl lowing change^* 




Everyone helped. "With.^ wooden knife on^of the .-girls took 
% some, paste out of the bowl and began^to tnRe a copy of her 
•school. * % + ' • _ ' * ■ 



The 'previous example cfin also serv$ .to illustrate other* Criteria* 
^fpr rewriting in addition t^p addressing jgrticul^r questions. .For one, 
by introducing an explicit topic, it amends a lapse in the cohereiJbe 
of the 9riginal passage, the third paragraph of ' whiche began with the 
cited sentence* Thi* paragraph essentially! recounted the girl's:* 
, activities as* well as* the activities of kn unidentified 'boy. *!These events, 
though ultimately integrated, have to .be lookpd^on as elements of story 
detail subsidiary %to aSa unstated topic. * 

A second purpose of rewriting; to increase the cohesion, of ^passage • 
'p&ttoces, is illustrated in the present example by the- logical inclusion 
of. the stated referent 1 within the sc&pe of a more inclusive entity. The 
mention of everydne in the Introduced topic is 'synonymous with a 
-V^yeviously m^Ationed reference to Ruth 1 s ^friends.' The girl anti the boy/w 

who are subsequently introduced, are themselves uilderst<M)d as. members of 
■ . the larger class of s friends. The original version did not ignore this • , • 
Relationship, but^it left it to be inferred entirely from a connection 
in the .preceding paragraph. * / w 



The Rewrites / . ' j *« 

Although some test questions were spree if ically addressed in the 
^rewrites, for the most part changes were made -without regard to the 
^information being tested. The larger considerar^n was to construct 
a cohesive anij^coherent account of the basic* passage information. We 
hypothesize?*! initially that a passage- rewritten to eliminate incon- 
sistencies and to clarify'the progression of information would enhance 
readability, make the task of reprieving specific information easier, 
and ultimately make test questions easier £o answer, whether- or not they 
^d been targeted f6r attention in the. rewrites . (In a few instances 
fhe adoption of this priority, in fact, led to an unintended complexity 
fot the student on particul ar qu estions (see below).) The parages, 
consequently, were regardecrs^s whole and entire elements and whtether 
or not a specific question wasNtffected by rewriting was in part. a / 
factor of circumstance. Many questions, in fact, were judged not to^Jf 
have been affected by the rewriting and, in the later analysis per- * ' # 
formance on these questions was ^computed sep&ratgly *f ronv t>hat dn«*other 
questions where the rewriting h^d^ro^clfi a 8 d^soe^lble-Klifferenc^. , 

. . . * - • 5 * 

As mentioned earlier^ tha .'£ype*of alterations*. made; in" rewriting 

passages dtra not fundamentally' change the character of the original m 

* .* 9 
versions. They did, .however , have an efffect in changing the character 

of the syntax.'' 1$ general, the changes were not^ sensitive, to the usual 

readability considerations of keeping sentences short and uncomplicated, 

especially for younger children. In- many cases, where it was judged 

the sentences were""collkps4d into a single, usually logger version. 

A -check on what ef f ecfrStt^S^had on computed readability (using, the Fry 

formula) showed that 7 of 14 rewritten passages ^increatfed ttieir 'scores 

by more than one full grade level. v Other passages were*iot ap*fc:iably 



affected ^(Figure 4-3). These findings- Were within our expectations. 
, because the increased .^ngth in some passage sentences' directly enters. 
* into the Fry readability equation. Overall, however, the sentences of 
the rewritten passages showed a greatar variability in length 
'than did^CJJ?^entences In the original version. 

'. V 



> Figure 4.3 

Readability levels for 'original and rewritten passage's- using the- Fry formula 
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This was not a planned goal, but it emerged \iatu3fally^f r6m attending % Jto 
other considerations. . 

V . ■ \ . ; 

Types of ChgggegjCadg 

Most of the changes made fell into five main types: 

(1) reordering of passage information * t 

(2) restatement of passage information * 

c -* (3) specification of inferrable information - 
] (4) clarification of paragraph topic ^ 
(5) v attribution of inforjnation 

Reorderings produced th^ most dramatic changes, in that'the floy 
of information could 'be substantially affected. 'Most instances of this 
ty^pe of change .were made in order to*restructure the scope of a topic, 
either in its rel^ion to other topics within the passage or. in its* 
relation to .the detail in the passage* The first 9xibtype is illustrated 

by the Lawn passage^(ITBS 7) which begins > 

# * 

In order to make it grow' well, a lawn^houid be watered 
the right way. A Strong hose spray is bad fW a lawn-. 

These' -two sentences in the full^context of the passage Jre both t^ici 5 ;, 
the full detail 'includes ^ discussion of good watering practices and 
poor watering practices". " Consequently, tfie two sentences were separated 



:es we 
jprom- 



as the heads of different paragraphs and the details appropriately . 
allocated,. The. first, paragraph began 

- y ' • . \ - f 

In order to jnakt a lawn grow well, ilt should^be watered * 

- • < " ; 

correctly.. TJie best way is to.^pray or sprinkle the lawn. 

I 

and the s6coiht-\ , * 

, A strong hosK spray is bad for a lawn. " The force of the 

water packs the^oil particles together.... 

s ^ ... ' ■ , 

The second t^pe of reordering amounts £o a~ rec^tegori2ation of 
• * t f 

passage information, motivated by-a determination that certain pieces^ 
of, infoT^tionjJj^e ordered^in the original in a way which masks their 
true discourse' relationships. In the Pyramids passage /CAT 18D)„ for 



instance, the concluding sentence of the passage suggests that in that 
position it is acting as the author's assessment. In truth, it may 
serve as sufeh, but more importantly it^slmply recapitulates the. essence 
of the discussion and so, has more weight; as a topic than, as an assess- 
ment. Consequently, this sentence, with minor alterations, was moved 
to the head^tff flNL second paragraph where it organizes detail describing 
some physical characteristics of the Great Pyramid. It begins 

It has been determined beyond reasonable doubt that the 
* Great Pyramid was built more, than 4000 years ago and th>at 

its builders yere advanced in science, mathematics, and ♦* 
architecture. They constructed its four triangular sides 
. 3 to a height of a 40 story building.../ * 

m o Continued research and -study may offer a better picture 
of what the Great Pyramid is> but it still does not tell 
us why or by whom it was built- • - 

Restataaent of passage information is. of ten done in conjunction with 
ot.he* changes in order to adapt the altered information^to. new locations 
or new functions. Other times it ^s done to better ada£t a point to its 
contextual environment. For instance, in the Plants passage (CAT 15D) 
a discussion of how two varieties 4 of insect-eating plants, capture insects 
b'y moving their leqpes is foJ.loH^d by a paragraph describing how a third 

variety accomplishes this by another mechanism. In the original the 

» » % « 

paragraph begins by Siting a fragmented list qf facts whose relationship * 
to one -another % im problematic. 

• - . *; i 

' The pitcher plant "feats insects, too. It cannot move its leaves 
the^ay the other .insect -eating plants do. The leaves of the 
pitcher plant make «a sweet smelling juice. Insects* come clos^ 

to the .leaves to taste the juice. - , \ . » 

* 4 %m 

In the rewrite, the relationships, are made more apparent by restating 

the information, in *the process, collapsing the first four sentences into^ 

two and 'introducing cohesion elements ^giich iJind Che information. 



The pitcher Riant also captures Insects, but not -by moving 
its 'leaves. It makes a sweet smelling juice which draws 
insects to it. ... • 

In some cases a restatement is made to eliminate an ambiguity or 
an imprecision. In the Ellen . passage (ITBS 7), for instance, the 
second paragraph has a teacfier discovering why Ellen couldn't spell well. 

^ At last the teacher found out why. She could not se$ the ' 
- chalkbo&rd .well from her seat. 



The she beginning the second sentence Is potentially ambiguous in its 
reference.' It logically would seeqj to refer to Ellen, but coming as it 
does with a mention of the teacher intervening, it could structurally* be 
identified as the teacher, rather than Ellen. ^This interpretation would 
preserve frfae referent focus, something which the use of the-i>rcmeun would- 
suggest ah yway . I*L£™ e rewritten version the ambijguity is eliminated by 
eliminating the pronouns and restating the intended referent* 

At last the teacher found that Ellen could not read the * 

chalkboard from her seat. 



The third, type of change speci^i^s information which was left to beJ^^ 
inferred in the. irrlginal versiqa. since a requirement to* draw an inference 
is in itsel'f not unreasonable and is, in fact, a necessary cdtqponent 



of reading comprehension, only those structures which were felt «o be 
imprecise or unnecessarily vague were singled cut for rewriting. A 
particular example is Included In the Minerals passage (ITBS 8) in a 
section li^Lng a number of ways to maintain stores of minerals. Th&- ^ 
third point is stated as follows. 

Three; related to number two, is to find cheaper methods ^b* 
* ~ £ mining those deep-lying minerals and of proceskirife otes « 

minerals. 

The second point, mentiohed parenthetically , is to explore for deeper • 
'mineral sources. Consequently, the nature of the imputed relationship 
between two and three is* causal, ftather than leave this inference as 
unnecessarily vague as it-i?,* the sentence was rewritten to read 



\* of min 
» low in 



Three, which is necessary if two is going^to be sixicessful,^ 
« 

is to find cheaper 'methods.*... _ - 

The fourth type of change, clarification of the paragraph topic, « is 
related to the preceding type.- Both involve a greater specification of 
presupposed information, but topic clarification has more important 
functional consequences, since the inference involved ca^ be critically 
important in schema formation. The Bricks passage (MAT Primary 2 JS). 
illustrates this point. The original passage, in a single paragraph, 
discusses something of the historical manufacture and use of bricks 
and concludes with a reference to the modern use of bricks. 



Altnough they are now made by machine, bricks continue to be 
important in . the building business. ^ 



( 



In the middle of this paragraph, the information abruptly changes in 
scope from a procedural account of brick making to^ giving three instances 
of the usefulness of .bricks. The three uses are not linked to any explicit 
encompassing statement in the passage, leaving them with only an inferred % 
topic. - In the rewrite^, fcfie hiatus* between, the two* topical foci, was 
made explicit by* pxtef^g eack in a separate paragraph and making the 



* 



'topic of the second paragraph explicit. The rewrite in part read 

Brick houses had many good points. If they were built on 
a strong ba*ee, they? could stand for hundreds of years. 



where the first sentence is^the inferenced topic made explicit. 

In this same passage the concluding sentence also implicitly 
recognized a. separate topic in its mention of machine-made bricks. This 
topic tfas also reified and set-up to head a separate paragraph. 

Bricks are still made today, although now they are made by 
, machine. They continue to be important in the building business. 

This paragraph even as ,it stands in the* rewrite' is not completely | 
coherent < since it l^cks the points of detail contrasting machine-making 
with hand-making which would make a tie to the final. assessment taore 
motivated. * • 
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The last type of change, attribution of information, will perhaps 
^ be more controversial than the others because, by *its intention, to 
clarify the source of information, it may arguably change the* author's 
ixitent to leave the attribution* Vague. - This .type of change was for the 
most part made in passages with a persuasive purpose, where the reader 
was asked to assess the validity or f activity of some claim. This eptire 
genre of questions, however, lead^" to difficulties on the reader's part 
because questions may be answered differently as one o? another point of view 
is adopted. The aim in rewriting the passage, then, is to eliminate some 
• of the ambiguity by making the point of view clearer. For example, in 
'the Andrew, Warr ell letter itkT 15C) the statement 

It's the tyjst show Oliver School has ever had. 

T6aVes~ it uhcITear if this is the Tett^^wKt^r f s~ p^r' sdnal op iniott or his 

reporting of the student body f s con&ensus. To clarify the situation, 

i 

the^sentence was rewritten* to include the pronoun we referencing the 
author and the * club he is noted as , representing. 

We think .that it's -the best shbw*01i.ver School* has ever-' 
had • • • • 

The reader must atill infer the identity of the we, but* the likelihood 
pf it referring to the studert body is^lessened. 

• Effe ct of Rewrite on Question s • 

— y ' : ~* < 

After all passage*, h ad „ been rewritten a .determination was made far 

each question as to whether the rewrite had affected the *answer4bility of 

a question in any way. Underlying this procedure was a desire to 

distinguish the composite results from the set of results on the affecte^ 

v questions. The larger purpose was to provfde some gross indication of 

how the cumulative affect of rewriting a passage differed from the question- 



:t of rewriting a passage di: 

specific effeets,* whether, for instance, the fcewr^te as a whole 'made -it 

' * f * . # , 

easier for the sfcudent to locate information Televakt to answering even. 

uxlaffected questions. ' p «* , : , 1 

The problem was complicated i-t the outset by the difficulty in ] 

deciding whether a particular .passage change had . indeed altered the 



* probability that 4 question might be answered, correctly. Some changes which 
had been made^for reasons, other thpa their impact on a Articular question 
could nevertheless exert a subsidiary effect on that question. For 
instance, a reordering of information in the Bill Cosby passage (C4T 13D) 
for the purpose of 'clarifying the topicrdetail relationship possibly 
altered th£ interpretability of a test question. The question* read 

Bill Cosby T s stories show that he probably was, 4 
, a. happy about, being young 

^ b. excited about watching TV - " 

„ • > 

: C. worried about his brothers > 
d. curious about making records 

with the expected answer the first option. The original version cued 



the answer with the following sentences, cpmprisiijg the last paragraph * 

k « 

' of the passage. , x 

Bill Cosby likes children* and they like himl His * * 

* stories help everyone remember what i^^vas.like to be 
young. * 

The rewrite puts Cosby f s stories, the passage theme, into topic, 
prominence as follows. * 

Bill Cosby 1 s stories show that he likes children and 
the things they do. They help everyone remember what 
it was like to be young. ' 

The required evalua::'.ve demands of the question have not ^jidfcged, but the 
match in wording between th6 question and the rewritten passage is closer, 
perhaps making it easier tor k student to draw the necessary inferences. 
'Because of this possibility th^g question was considered to be affected 
by the rewrite, even though only minimally. 

Judgments on the effect of a rewrite wefe 'subtle enough in some # 
instances that two raters were used to independently assess the effect 
on each question. Initial differences in judgment, which occurred on 
approximately 15 percent of the questions, were resolved, in conference 
to the satisfaction of both raters. . 



In a few cases it happened that the rewrite had a negative effect on 
the answerability of a question, though this went undetected until after 
the experiment' had been run. For questions so affected the rewrite 
had inadvertently made one of the distractors a "logically po3slble answer, 
in addition to >an<f in competition with the expected answer. l\ the ' 
Ruth Asawa passage (CAT 13C), for instance, a ^change in the second, para- 
graph for the purpose of making the topic more precise ende d up high- 
lighting one of the distractors. The original version ^^an the para- 
graph as fdllows. 



First she drew a map on paper. Then she and her friends began 
« working*. » 

Since the drawing of the map. itself constitutes work, it should fal} 
within the scope of the second sentence. The rewrite then attemptf to 
restate the scope of the work by more clearly specifying the sequence 
of events. It took the following form. - , 

, 9 

She called her friends together and they began workfng. First, 
Ruth Asawa drew the map on paper. » t 

> 

One of the questions on Jth& passage focused on the sequencing of. these t 
activities. It askecf ' ^ * " 

** ' V 

In this story, which of the following did Ruth As&wa do iiirst? 

a. " made the paste . « ^ 

b. ' made a -metal map * << 
• c. talked to the children 

d. drew the map on pa'per . , 

The expected answer, the fourtV alternative, is still overtly cued in the 
same way as in tKe original passage, namely by the word first. However, 
the rewrite "also makes the third option a possibility by stating that sfie 
called her friends'. ' Since this^ event precedes the m^p drawing, Students 
coirid ignore the overt, literal cue' and interpret this ^s an inferential- 
question instead. These students were, in fact, being unfairly penalized, 
for «their greater perceptivity^ . * 



A total of 9 questions (out of 85) were judged to be negatively 
affected by the rewrite. Because at one stage .of the analysis these 
questions were eliminated .from consideration, the rationale for' the 
decision on each of them is provided below. The first question, from 
tht Ruth passage was discussed above*- % * * # * 

* 2. Plants Ques. 60:' What happens* when insects land on the. sundew? 

Expected answer: /They stidk to' Its droplets. 

Highlighted distractor y They dtfcwn in its, juices. 

Rewrite: When an insect lands and gets £tuck on* one 

of the drops, the sundeVs "arms 11 close around it. 
^ Comment : This question was judged to have nO|J>est answer 
since the reader, would expect that the end point of the 
process would be the targeted answer. This was not included 
among the options. The rewrite rather than clarifying the 
situation aggravates it by implying that the process of ^ 
getting stucjc i& presupposed background information ratiher 
. 'than the looked-for result. The highlighted distractor . is 
an inferrable consequence of the insect landijig on a Hrop 
q£ fluid. 

* • <t 

** * *• 

3^ Sailors Oues. 38"T Why did the sailors become tired and weak? 

Expected answer; they Mid not eat the right Vood.* 
* Highlighted distractor : *They did not have enough to eat. 
Rewrite : ^ey^ pould become so tired and weak they could not 
work.* The, men in charge did not know what could be wrong. 0 
They ruled out food b.ec^se the sailors got plenty .of meat « 
and bread to eat. W ; *' 

Comment : T^e last sentence quoted above leaves open the 
possibility under ope interpretation that* the men in charge 
eliminated (ruled out) food, implying tlfot/the sailors didn f t 
. have enough of , the right food to eat. 

4. Andrew Oues. hh: Which of the following is a fact about the 
"special prize"? ' 



88 r - * } 



0 



Expe cted answer: Andrew Warrel wants the- club to receive 

' " 4 

the prizes v 

. Highlighted -distrac tor : • Oliver School won the prize for 
their decj>^ted halls. V*'* 

R^rite: In fact, we f re f hoping that the Pen and Ink Club 
will be given the school's ^special prize. 

Comment : Because the prize was definitized in the rewrite, 
sgme readers eyidently came to interpret it as something 
already in existence rather than as. a hypothetical prize. 
The highlighted distractor is then the only plausible opftion 
which could constitute a fact about the prize. 

5. Lawn » Ques. 60: Which would be tha best oame for this article? 
Expected aftswer : "How to water *a lawn" 

f w 

highlighted distractor : "A growing lawn" 

Rewrite : In order to make a lawn grow well, it should be 

watered' correctly. 

Comment : Evidently the juxtaposition of lawn and grow in 
the first sentence of the passage cued the distractor. In. 
the original version the first sentence vjlixtaposed lawn and 
watered. . „ * 

— ~~ ■ - * *• 

r 

6. Sequoya Ques.' 66: Whfen the Cherokees 'moved to the West, it 
was harder for them to talk together because 

Expected answer : they lived farther apart. 
Highlighted disETactor : theyjiad to learn a new language. 
Rewrite : The Cherokees, were pushed from their homes iti the 
South and forced to scatter to the West... Sequoya- knew it 
•would be Hiard Tor families and^friends to communicate with 
. each other. The Cherokee language^had b.een spoken for 
centuries, but thje people did not have a way to write It. 
Copment : In the original passage the information is given 
that the homes of the* Cherokees were scattered probably 
making it easier to infer the expected answer. In the. absence 
of this clue in the rewrite, some readers evidently chose to 
interpret the information that the Cherokees learned to write 
as implying that they Had to learn a new language. 



7. Maria Ques. v 34: Before Maria* Mitchell could claim to have 
discovered a cothet, she had to 

Expected answer : make a series of observations. * * 

Highlighted distractor : calculate how far the comet' 
from the star Polaris. 

Rewrite : ...Maria noticed aift unusual, fuzzy white patch 
near'the star Polaris. She suspected that this might be a 
major discovery, and for ggveraf nights she and hef father 
j observed the patch as it Bedame clearer. 
Comment : The rewrite reorders events so- that the usual 
observations' werA followed by a suspicion of discovery. 

# 

The original, however, removes the element of suspense and 
states immediately, "... while making her usu£l observations 
Maria made a major discovery." .In the rewrite, then, the 
association between the, proof-positive discqvery and the/ 
observations is not as apparent and readers were freed- to 
infer a reason.- in the highlighted distractor. 

♦ # 8. Australia Ques. 88: What is the main purpose of the second 
paragraph? - - 4 ^ 

Expected Apswer : To tell how the glider flies. 

x Highlighted distractor : To compare the glider with a bird. 

>. * * 

Rewrite : It dqes not flap its membrane like a bird would . 
flap its wings.* . 

Comment : > In filling out the details of an analogy 'which the 
original leaves only loosely specified, the rewrite seems to 



make an implicit comparison more consequential* 



AT- 



9. Australia Ques. 89: Which of these phrases is closestr to 
the meaning of the word "launch" 4s used in the -second * 
paragraph? * # • • • 

Expectep answer : To shovfe or send off. JB^T 
' Highli mted ^i istractor : To set afloat. « ^ 

Rewrite : :..it launches itself from a high tree and sails. 

Comment : The original version specified that the glider sails 

through tfte air, appoint which being absent in phe rewrite sets 
* aip the possibility of floating rather' than a directional motion. 

* ■ 



All in all the probability N that the rewritten passages disposed 
readers to other than the expected answer^ points up if nothing else 
the difficulty in designing -questions to be unambiguous. It offers 
a lesson in humility, ahd certainly must put one in greater sympathy 
wi^h the test developers. Fortunately, However , * the examples can *be 
used to test the interesting hypothesis fhat readers a're in^fact 
sensitive to such subtle' changesjfcn structure. Specifically, we * < 

would expect that the response pattern on the items would differ 

between the original and rewritten, versions , witfi the rewrftt^n * 

% to H ' 

versions showing thfl^f f ecu of highlighting one of the distractors . 

^over the, other options. In the original versions there would pre- 

sumably be uot as much reason othex thai; chance for the highlighted 

t> * . ~ f: * * 

distractor to be selected.^ m **/ 

As the data tabulated in Figure 4-4 dSmonstrate , this expectation • 
is met. All of the questions, except the one from the Ruth passage, 
show a reduction in the percentage of correct' responses, with the 
reduction accounted for mainly in increased response to the highlighted 
distractor. Furthermore, the pattern of incorrect responses changes 
in the predicted ^difecti&n. In the Ruth passage, for instance, 
distractor ^ which was favored 40% of the time in the original was not 
selected at all in -the rewrite. , On the other hand the highlighted 
distractor was chosen in the rewrite but not in the original. Altogether 
6 of the 9 questions show a similar shift in response 'pattern,, and 
2 -of the remaining 3 show an increase in the percent response. for the 
. highlighted distractor. Only the Lawn passage question shows essent>ally 
? the same response pattern between thje two versions. The data, 4 in short, 
seem to support the hypothesis of reader sensitivity £q structural 
changes/ In the following section we will present the remainder of 
the experimental results with the aim of* providing further substantiation 
• for Jthis hypothesis. v * ^ . 



Figure 

Questions for which the rewritten passages negatively affect the probability 
of select ing the expected answer. The item analysis presents the percent 
of the low achieving students responding to each option. (The Distractcy 1 
heading for the original passage 1 corresponds to the option listed as 
Highlighted distractor for the rewritten passage. The distractors listed 
as 2 and 3 under each category likewise correspond.) { 
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Analysis of tefft data * 

Before beginning the analysis of the experimental test data, 
it should be remembered that the passages used wer 4 e purposely selected 
for their diversity of purpose and type. This had the effect of 
making the experimeotal test conform .more closely to a typical readin'g 
achievement test and permitted a variety of test objectives to be 
examined, but it also had the effect of limiting th£ comparability 
of passages.. In f? ct ^ few individual passages were balanced across 
all tfce objectives and some tended to focus Almost exclusively on a 
single objective. Then,, too,, even though we 'put no great, reliance on 
the supposed usefulness of readability f grmulas^to assist in making 
decisions ^about passage difficulty, certain rewritten passages may 
Issibly ^introduce structural complexities beyond those present in the 
riginal Versions. In a^ sense we may have achieved a reduction in the 
complexity of the discourse structure of the test passages at the 
expense of introducing greater syntactic complexity. The effect of 
this kind of trade off is predicted to benefit comprehension overall, 
but it could happen that the absolute size of the predicted gain is 
tempered somewhat by the increased syntactic ^omplexi^y of the rewritten 
passages. The experimental design however, was not sensitive enough 
to decide this matter, although there^ is some evidence, to be 
discussed later, that .the greater syntactic complexity may in fact have 
had some depressive effect. * 

The unresolved question of the effect of increased syntactic 
complexity coupled with the small number of subjects and the large 
number of structural variables inflGfkiced during rewriting complicate 
the statistical analysis of the,<Jat4; making it improbable that gains 
or losses large enough to demonstrate statistical significance would 
be achieved. Consequently , the meaning of any particular statistic -* 
must be qualitatively assessed by considering the probability of the 
gain (or loss) in measured comprehension' against the potential influence 
of unmeasurab]^ competing effects. Admittedly, .this is not as • • - 
.satisfying an analysis as wotild be 4 demonstration of statistical 

s . 



93 r * : * 



significance, but at the least it can provide some indication of the 
likelihood that tighter controls and the inclusion of more *sub jects 
would produce the desired effects. 

For ease of comparison and analysis the cumulated scores of all 
subjects responding to particular qtiAtions are.-presented in figures 
4-^,b^c together with information characterizing, each question- for 
its declared and actual comprehension objective and the effect the 
rewrite had on the answerability of the question. In addiction, the, 
discotfiFfe type of each passage is provided. It will be recalled 
ftom Earlier discussion that the declared comprehension objective is 
that w^ich is specif ie^ in the respective test manual, while the 
^actual objective is that which was determined by our own analysis. 
The effect of rewrite has three possible values: positive in the 
case that a change had a predictably enhancing effect on comprehension, - 
negative , if a predictably depressing effect, and null if the 
particular question was unaffected, * 

As the three tables show, the results on individual questions are 
equivocal, other than that' they verify the proficiency criterion used 

i 

to selecp subjects. In other words, for all but a very few questions 

t£e low achieving group scored lower 'than the high achieving group on 

all passages.* However, a fcest of significance for the difference in 

r 7 
^proportion of low and high achieving students answering* correctly 

indicated that thfe differences were not statistically significant for 

certain passages in their original versions (see Figure 4-6). Since 

w$ can assume these differences to have been significant in the norming 

sample, we have a^me numerical verification of our claim that the 

subject sample was too small to yield statistically significant data. * 

It will, then, be possible to show sign£f icantly^ enhanced performance 

on the rewrites only for those passages in which the difference in 

performance on the original versions was significant to s£art with. 

•If the measure of enhanced performance is taken to be a -meaning- 

r 

. ful redqction in the variability between the high and low achieving 
groups; .then we areT able to demonstrate such a reduction only in the 
cases "of the Bricks ^and Andrew passages (see Figure 4-6). On the 
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Figure 4-5a 
iry ahd Results on Experimental Items 
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f Figure 4-5b' 

Background Data Su—ary and Results on Experimental Itegg 
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.Figure 4- 5b (Continued), 
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Background Data Sugary, and Results on Experimental Items 
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Figure 4-5c (Continued) 
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Figure 4t6 

z scores for the difference in profiortion^of yt:6rrect' response fo^tWeeh 
high and low achieving^students on two versions of the tested passages 



3rd Grade 
Ruth - 

m 

Bricks 
Cpsby 
Ellen 



Original 



1.7 
3.6 
4.3 
2.4 



(n. s . ) 
(p<*01) 
(p<.01) * 
(V< .05) 



Rewrite 
1.5 (n.s.) 
'1.3 (n.s.) 
3.4 1 (p< .01) 
4.2 (p<.01) 



5th Grade 
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.1.6 
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(p<..01) 
(P<".01) 
(P C.05) 



1.9 (n.s.) 
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other hand it appears that using the same criterion the rewrite was« 
detrimental to comprehension in the» case of the Ellen, Sailors, and 



* of their significance between the two versions. These, results are 



Sequoya passages. The other passages showed no change in the level 

i the two v 
data in Fi 
lirestions o 



reiterated in the summary of data in Figure 4*-7a. giving the percent 
correct response across all qvrestions on a particular passage. It, # 
would seem* then, that the hypothesis that passages can be rewritten * 
to enhance performance holds in some cas^s and fails in others. 

However, before accepting this conclusion the data should be 
sifted to focus the analysis on just those questions which were 
affected by the rewrite. This was done in a two-step process, first 
eliminating the null questions (Figure 4-7b) and th^n, subsequently, 
all the questions judged to have a negative effect on comprehensibility 
(Fig. .4-7c). Following this procedure we realize two advantages : the 
number of passages showing a decrement in proficiency for low 
9 achi_evers ^decreases from six to four and^the variance in petcent 
correct response between original and rewritten versions increases for 
low achievers; 

This result indicates that much of the reason for depressed 
performance on certain rewritten versions is attributable to questions 
which were either not addressed in rewriting or were negatively 
affected by the rewrite.. What this conclusion further suggests, 
however, is that the rewrites, may have introduced syntactic or 
structural complexity of a degree which hindered th^vstudent in answering 
questions which had not tleen specifically clarified in the rewrite. 1 
If this is the case, however, the increase^ syntactic difficulty does 
not correlate simply with greater sentence length, since there is no 
correlation between an increase in readability ^fcore (see Figure '4-3) 
of a rewrite and greater difficulty. Notice, for instance, that 
the Pyramids and Plants passages, which shQwed the largest increases in 
presumed difficulty using the Fry formula are not^among the passages 
showing a decrease in proficiency. 

It should be pointed out that the analysis of data for the high 
achieving students does pot pattern the same way as the d&ta for low 
achieving students. Most importantly, high achieving students showed 
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Figure 4- 7b 

. Percent correct response; questions affected by rewrite 



3rd Grade 



Low Achieving , 
Original RewritS 



/ 



High Achieving 



Original 



Rewrite 
1 



Ruth (Q=5) 
Bricks (Q=4) # 
Cosby (Q=4) 
Ellen tQ=l) 
5th Grade 



56 
35 
41 
30 . 



•65 
.74 
54 
25 



80 
100 
100 

67 



84 
85 
100 

80 



Plants (Q=7)" 
Sailors (Q=6) 
Andrew (Q=5) 
Lawn (0=3) 
Sequoya (Q=6) 



64 
80 
40 
I 33 
50 



75 
67 
67 
47 
53 



90 . • 
97 - 

100 
100 

* 87 ~f 



96 

93 

84 
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100 
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71 
5& 
63 
45 



55 
67 
, -40 
' 52 
60 



90 
96 
100 
98 
80 



95 
92 
100 
.93 
88 



V 



9 

ERIC 



103 



i 'J 'i 



„ Figure 4-7c" 

* Percent correct response; questions affected positively by rewrite 



3rd Grade 



Low r Achieving 
Origina l Rewrite 



High Achieving 



Original 



Rewrite 



Ruth (Q-4). 
Bricks (Q=4) 
Cosby {0/S4) 
Ellen (Q-l) 



55 . 
35 
41 
30 



63. 
• 74 
- 54 
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100 
100 
67 



85 
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85 
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Australia (Q-5) 
Minerals (Q-5) 



44 
64 
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55 
71 
40 
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95 
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95 
95 
100 

' 95 
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depressed scores on certain rewritten passages low achieving 

students demonstrated great improvement on f the BricM and the Andrew 

passages, in particular. Where the low achieving- students 

Significantly improved their performance on Bricks (p *.05) and nearly 

so on Andrew, the high achieving students last 15 percentage points 

on each a non-significant, but troubling reversal from expectation. 

The reason for this discrepancy is not entirely clear. It may be a 

skewing effect of high achieving students performing at or near. the 

-ceiling level or it may be attributable to a real difficulty arising 

out -of the rewrite, presumably because the form of the rewrite makes 

it lesfe obvious that a question falls within a certain category. All 

except one of the Bricks ^ questions^ for instance, require literal 

comprehension. It* may be that the rewrite makes "the obviousness of 

this task ill the original version less obvious in the rewritten 

version. The ldfa Achieving students presumably estape the dilemma 

because the clues to the correct answer as in the original version 

are not as apparent to them as they are *^o the high achievers.* If 

this explanation holds it would indicate another difference in 

comprehension strategies between .the two groups,' the higher achieving 

students acting as experts wh'ose attention is directed to points 

of information which in their oddity of placement stand out more 

prominently. The low achieving students because they are not expert 

miss seeing these points of information as out-of-the-ordinary and, * 
t 

in fact, are misled by the poor discourse organization into not seeing 
them at all. Any improvement in organization, consequently, is likely 
t<5 benefit their comprehension. 4 

Although most of the analysis was confined to individual passages, 
a cross-passage analysis of reading objectives usin^ the breakdown 
arrived at earlier was undertaken. The rationale for this analysis, 
while not deriving from passage comparability, was assumed to derive 
from the comparability of questions asked of these passages. Questions 
purportedly testing -similar objectives set up similar expectations in 
the reader's mind and possibly activate similar search procedures. 
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Some evidence for this claim was provided^ in discussion , of the t 
interviews . 

Figure 4-8 provides the results of data analysis on the experimental 
* test .results, taking as an independent variable the actual comprehension 
objective of each test*item as tabled, on Figure 4-5. For each objective 
the number of questions exemplifying it is included in 'parentheses ^ 
following its listing. . ' 9 

Figure 4-8 is of interest because it provides some greater 
understanding of what was accomplished by the rewrites. It 9hows, 
first, a difference between grades on the relative proportion of 
questions in a particular category — almost twice 'as many literal 
comprehension questions in the 3rd grade as in the 5th and 8th 
grades . For the two higher grades proportionately more weight is 
given to inferential and evaluative questions. This difference is 
of some consequence because we also find that literal comprehension 
questions seem to have benefitted substantially more from rewriting 
than the ot!her types. In fact, at the 3rd and 5th gfcades improvement 
in literal comprehension comes very close to reaching significance. 
This would indicate the positive effects of attention to passage 
organization to be more apparent in the early grades. Obviously, 
how£*fcr, this claim must be verified with larger samples, since the 
majority of 3rd grade .literal questions were drawn* f rom a single 
passage. 

"^^ftfc^-mest disappointing finding revealed jLn Figure 4-8 is the neg- 
ligible or even detrimental effect of the rewrites on inferential 
questions at the 5th and 8th grade levels. Only the low achieving 
3rd* graders showed improvement on this category. Why thts pattern 
f emerged is problematic. It may indicate that the nature of the infor- 

4 mat ion organization in the' higher grade passages was not as susceptible 
to change without violating' the stricture on changing or importing 

\ information. In this event 4 the changes made would necessarily be 
trivial ox inconsequential. Evidence to verify this point, however, 
is inconclusive, as a judgment regarding the relative effect of a 
change is somewhat subjective. 
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Figure 4-8 

Percent correct response by question objective : 
questions affected by rewrite . 



3rd Grade 

literal (Q=5) 
inferential (Q=5) 
main idea (Q*2) 
evaluative (Q=2) 



* * Low Achieving 



Original 

• . 48 
39 
, 56 
■•» 36 



Rewrite 

77 
58 
60 

~30 ' 



High Achieving 
Original Rewrite 



100 
92 
70 
70 



88 
92 
80 
90 



5th Grade 

literal (Q=5) 
inferential (Q=13) 
main idea (Q=2) • 
evaluative (Q=7) 

, 8 th Grade 

literal (Q-3) 
inferential (Q=6)# 
main idea' (0=5) 
evaluative (0^8) 



• i 

- 58 
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60 
'* 56 
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. • ; In other respects the results do show a rise in performance for 
evaluative!' questions at, the 5th and 8th grades for low achievers. 
This is noteworthy because the changes made to clarify many of these 
questions were relatively uniform. They tended to impart a greater 
specificity to the point of view the reader was to assume in answering. 
The results at the 3rd grade are inconclusive. The decrease may - 
simply be attributable to having oply two' questions in the sampltf^ 
or it might be due to* a real inability of 3rd graders to pejjform the 
kind- of synthesis of passage and background informj>f6n required of 
an evaluative question. 

All in all the * results of this part it the investigation provide 
qualified support that rewriting test passages to improve discourse 
organization can effect an improvement /in comprehension for low 
achieving students. The actual gain in achievement, however, seems 
to be .dependent' on the 'type of passage being manipulated as well as 
on th^ type of question asked of thi passage. The magnitude of the 
gain, whi^le significant iri only one of the tested passages, must bd 

-understood to have been tetapered by the small size pf the subject sample 
and, more importantly, by the decision not* to t change the passage 
questions asked on the original tests. We demonstrated earner that 
many of these questions exhibited faults in their construction over 
and m above Whatever faults could ^e attributed to passage organization. 

.The implication is that 4?y designing better reading passages- and . 
better sets of questions, low achieving students would exhibit more 

4 

evidence of their real grasp oh comprehension skills, and less of 

4 * 

their lack of test-taking and metacomprehension skills. 

( 



t 

* , Chapter 5 < " , 

. * SUMMARY AND CONCLUSIONS • ' , . - * * ' ' 

The evidence of the interviews and, experiment al test , is strongly 
indicative of two major conclusions. First, poor reading ability as 

m 

' determined through standardized test administration h&s Tiot as much 
to do with deficient comprehension skills as it does with deficiencies^ 
in test-taking and metacomprehension skills. Popr .readers of the 
caliber we looRed kt in this study were typically able to find their 
way to plausible but incorrect answers through the application of. 
comprehension skills which were qualitatively no different from those 
used by better readers. Both types of readers adequately demonstrated *■ 
their ability to extract explicitly stated factual information, to 

4 

draw inferences, and to make evaluations of passage content against 
their background experience. Differences on these skills when they * 
were recognizable were attributable either to specific rather than' 
genetfic comprehension deficiencies, such gas not understanding that the a 
deitic word 'this 1 can point to a preceding rather than following 
sentence, <5r to an overreliaoce on imported backgrounds-information to 
justify answers. Both pf these characteristics, however, amount to 
quantitative rather than qualitative differences between the two 
populations 'and wotfld argue for greater attention being paid to 
discriminating students on their approach to reading and their strategy 
toward problem solving than on their ability to perform* basic cognitive 
operations on written material. 

The second conclusion is in P art ^ generated from the argument of 
f the first. If good and poor readers .are not so much to be discriminated 
on the ba^is of their comprehension skills, then the 4 tests which 
purport to measure these skills are themselves deficient in not ^ 
suggesting the' real reasons lying behind performance variability. 
Moreover, the fests are open to criticism in that sojne of their 
questions permit two or more logietfUy defensible choices. Since we 
anticipated the possibility of multiple interpretatibns from a linguistic 
analysis of test items and verified these ambiguities through the 
interviews and experimental testa, we can conclude that "the tests, if 
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their intention is to produce items* with only a single defensible choice, 
have failed in their intention. We have laid the blame for this at 
two doors: at the test-maker's since their inattention to the linguistic 
structure of reading passages creates the opportunity for multiple 
interpretations and at the test^takers' for not understanding how to 
recognize and resolve ambiguities. It is a real probability,* however,' 
that the test-takers' fault is at least partly to be jhared "by 
educators who do not themselves recognize children's true difficulties 
with reading and tiry instead to remediate problems which do not exist. 

Even if it •appears unlikely that standardized tests discriminate 
stTudents on the basis of their comprehension skills, the numbers do 
indicate that they discriminate students on some variable and, more- 
over, that they do it reliably. We have already said 'that these 

jprue discriminants are, brbadly speaking, metacomprehension -and test- 

\ * * 

taking skills. In particular, it would appear that the main difference * 

between good and poo£ readers involves a reluctance or an inability 

on the part of poor readers to process information hierarchically. 

Given that the content of the selections students'read on tests is 

r 

generally unfamiliar, hiferarchical processing - a stragegy which 
organizes and assigns priorities to passage information - is probably 
*more effective in aiding information r^jrieval than a processing 
strategy which assigns equal weight to all passage information. The, 
latter what we have called dfetail processing, Is presumably" better 
assumed when a reader already has an elaborated schema on the passage 
topic. In that event tfte further details of the passage information r 
find their proper importance through' the implicit interaction between 
the inplace and the incoming information. 

Given the particular demands of the reading test situation, it 

; 

would seem, then, that it is a mistake for students to adopt a detail 

<+ 

processing strategy. It. may assure that some items are answered 
correctly, but overall it complicates the retrieval process because 
^information is difficult to locate on lookback. It is possibly for 
this reason that the poor readers we investigated were generally »un- 
willing to verify their answers. It would have been a time consuming 
4 task, possibly not worth the effort* in any evej>t.~ 
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The etio^LAgy of poor readers' preferences for processing written 
tefcf material' is a matter of some speculation. In the minds of some 
interviewed students, it seems to precipitate from a feeling that * 
tests ajid, reading are somewhat caprlcipus. You can get them to mean 
whatever you want, and the choice of a, correct answer is attributable * 
.to the insider's ability to crack the system. They may in fact -be 
fright on tjhis point, but that still leaves it problematic how this s 
defeatists attitude arises, especially since they have the necessary 
cognitive "raw material" to perform adequately. 

Perhaps one place to look for the answer is in the tests them- 
selves, which contrary to test-taking strategies which 'are advocated 
in instructions and by teachers, actually encourage detail processing 
by. asking for trivial and often times unhighlighted information. Such 
practices encourage undu^ attention £o these points of information and 
make them appear important not so milch for their intrinsic value to 
the pass-age content as for their extrinsic value to ^he test-makers. 
Unfortunately, this convention is adopted early on in the child's' 
exposure to tests, perhaps £n t;he mistaken, belief t?hat young children . 
are unable to cope with inferential or evaluative questions. Literal 
questioning becomes the standard for testing in the early grades and 
in all probability indflfces the detail pro?essing reading strategy 
which later' so inadequately serves the maturing reader. 

In consideration of what can be done to improve the situation, , 
the t study investigated what effect rewriting test passaged to make 
them 'better conform to theoretically ^determined standards' of good 
discourse structure would have on enhancing performance.. 'As diseased 
in Chapter 4, the results were somewhat equivocal, although overall 
the principle was^demon^Jrated, "especially if mitigating factors were 
considered. These included research design deci^pns, such' as not 
•altering the form or substance of certain •recognizably poor test 
questions and no"t\ importing information into test passages, as well 
as evidence that the discourse type of a passage makes i£ more orr less 
susceptible to improvement, especially in consideration of^he type 

of questions asked of it. • " 

» « « 
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Besides these controllable aspects of the;ptoblem, however, there 
is a set of subject dependent variables, such a£ motivation, interest, 
background knowledge, and preferred problem solving strategies, which 
must also be considered, to the extent thet '^low achieving readers 
are deficient in one or more of these matters' and recognize them- 
selves to* be, it is likely that its , negative effects on test per- 
^formance.have been carried through on both the original and rewritten 
versions. It is hoped that the rewritten versions in their greater 
clarity of presentation may have made students less apprehensive and 
more sure of their judgments, but evidence of this was not adducfed. 

Assuming for the sake of discussion that the hypothesis holds 
by virtue of acknowledging the* persistence of thes6 discriminating 
factors, then the experiment yields some information that improvement 
is mainly limited to requeati for information of a literal or ^evaluative 
nature* Inferential knowledge seems 'less susceptible to the kind of 
clearly demonstratable improvement as exhibited for the other types. 
The implication for the testing of reading skills per se is that ^ 
• inferential processing appears^ less dependent on the stylistic town 
a ^>assage takes.- This conclusion indicates that test developers would 
more accurately be assessing reading skills if they* confined their * 
attention exclusively to inferential questions. 'It max be noted in 
this connection that this is what cloze tests, such as the Degrees 
o'f Reading Pqwer, already do. y. 

Testers might object that a /est ignoring literal an^fcvaluative 

skills mioses the opportunity to inform teachers on important 

if* " 
components of their reading curriculum. As well-intentioned as this 

concern might be^ it must be admitted in light of this study's 

results that fcKe breakdown of ^objectives on standardized reading 

comprehension tests has no jfrowei K to inform on the true status of a 

student's abilities. Standardized test results cannot be used As 

the basis for a diagnosis of achievement, other than in the broad 

sense for whiqttf they are desi'gned, that being to make predictions of 

future academic success. , • 
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Since the Inclusion 6f questions asking ffir literal and evaluative 
information activates variables such as passage organization and 

background information, tests attempting to measure these skills 

• * 

introduce complications into* the^asgessment of overall reading ability. 
Reiterating earlier arguments, these skills include, besides reading 
ability, an ability to take tests, to process information hierar- 
chically, to detect flatos,*and possibly some others. A good reader, 
as we must understand th6 term from an objective assessment of 
standardized tests, is one who sttows evidence of more than simply 
deriving information from writing. *,He or she is to some extent an 
expert problem solver^, is well-informed, and sees things in the same 
way as the people who design tests. Whether or not this is how we 
should want to define a good reader is an educational issue^on which 
the choice of modifying our testing^ strategies or keeping them as 
they are should be based. 

.The most important , point to, consider in deciding the issue is 
whether the tasks^ we now expect stydents to perform and the labels 
we hand out* to students on the basis of test results have'a stul- . 
tifyii*4 Effect on students. Do declared low achieving readers who, 
in spite of their labels, know they-can Extract information from 
writing, come to believe finally that they cannot read and so fulfill 
the (fredictions made about them? If so, then this is our ^stftng^t 
argument for more j^refully considering the structural' quality of 
the written materials we present tcf, them and the real demands we 
make of them. * 
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